Weakly supervised fine-grained classification method of Alzheimer’s disease based on improved visual geometry group network

doi:10.11772/j.issn.1001-9081.2021020258

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 302-309.DOI: 10.11772/j.issn.1001-9081.2021020258

• Frontier and comprehensive applications • Previous Articles Next Articles

Weakly supervised fine-grained classification method of Alzheimer’s disease based on improved visual geometry group network

Shuang DENG(), Xiaohai HE, Linbo QING, Honggang CHEN, Qizhi TENG

College of Electronics and Information Engineering，Sichuan University，Chengdu Sichuan 610065，China

Received:2021-02-22 Revised:2021-04-28 Accepted:2021-04-29 Online:2021-05-12 Published:2022-01-10
Contact: Shuang DENG
About author:DENG Shuang， born in 1995， M. S. candidate. Her research interests include image processing， pattern recognition， artificial intelligence.
HE Xiaohai， born in 1964， Ph. D.， professor. His research interests include image processing， pattern recognition， image communication.
QING Linbo， born in 1982， Ph. D.， associate professor. His research interests include image processing， pattern recognition， video communication.
CHEN Honggang， born in 1991， Ph. D.， research assistant. His research interests include image/video understanding， restoration and compression coding.
TENG Qizhi ， born in 1961， Ph. D.， professor. Her research interests include digital image processing， pattern recognition， 3D image reconstruction and analysis.
Supported by:
Chengdu Major Science and Technology Application Demonstration Project(2019-YF09-00120-SN)

基于改进VGG网络的弱监督细粒度阿尔兹海默症分类方法

邓爽(), 何小海, 卿粼波, 陈洪刚, 滕奇志

四川大学电子信息学院，成都 610065

通讯作者: 邓爽
作者简介:邓爽（1995—），女，四川绵阳人，硕士研究生，主要研究方向：图像处理、模式识别、人工智能
何小海（1964—），四川绵阳人，教授，博士生导师，博士，主要研究方向：图像处理、模式识别、图像通信
卿粼波（1982—），男，四川简阳人，副教授，博士，主要研究方向：图像处理、模式识别、视频通信
陈洪刚（1991—），男，四川成都人，助理研究员，博士，主要研究方向：图像/视频理解、复原及压缩编码
滕奇志（1961—），女，四川成都人，教授，博士，主要研究方向：数字图像处理、模式识别、三维图像重建及分析。
基金资助:
成都市重大科技应用示范项目(2019-YF09-00120-SN)

Abstract

Abstract:

In order to solve the problems of small difference of Magnetic Resonance Imaging （MRI） images between Alzheimer’s Disease （AD） patients and Normal Control （NC） people and great difficulty in classification of them， a weakly supervised fine-grained classification method for AD based on improved Visual Geometry Group （VGG） network was proposed. In this method， Weakly Supervised Data Augmentation Network （WSDAN） was took as the basic model， which was mainly composed of weakly supervised attention learning module， data augmentation module and bilinear attention pooling module. Firstly， the feature map and the attention map were generated through weakly supervised attention learning network， and the attention map was used to guide the data augmentation. Both the original image and the augmented data were used as the input data for training. Then， point production between the feature map and the attention map was performed by elements via bilinear attention pooling algorithm to obtain the feature matrix. Finally， the feature matrix was used as the input of the linear classification layer. Experimental results of applying WSDAN basic model with VGG19 as feature extraction network on MRI data of AD show that， compared with the WSDAN basic model， the proposed model only with image enhancement has the accuracy， sensitivity and specificity increased by 1.6 percentage points， 0.34 percentage points and 0.12 percentage points respectively； the model only using the improvement of VGG19 network has the accuracy and specificity improved by 0.7 percentage points and 2.82 percentage points respectively； the model combing the two methods above has the accuracy， sensitivity and specificity improved by 2.1 percentage points， 1.91 percentage points and 2.19 percentage points respectively.

Key words: improved Visual Geometry Group (VGG) network, weakly supervised, fine-grained classification, data augmentation, Alzheimer’s Disease (AD)

摘要：

针对阿尔兹海默症（AD）患者和正常（NC）人之间核磁共振成像（MRI）图像差别小、分类难度大的问题，提出了基于改进VGG网络的弱监督细粒度AD分类方法。该方法以弱监督数据增强网络（WSDAN）为基本模型，主要由弱监督注意力学习模块、数据增强模块及双线性注意力池化模块等构成。首先，通过弱监督力注意学习模块生成特征图和注意力图，并利用注意力图引导数据增强，将原图和增强后的数据同时作为输入数据进行训练；然后，通过双线性注意力池化算法将特征图和注意力图按元素进行点乘，进而得到特征矩阵；最后，将特征矩阵作为线性分类层的输入。将以VGG19作为特征提取网络的WSDAN基本模型应用到AD的MRI数据上，实验结果表明，仅使用图像增强的模型的准确性、敏感性和特异性分别比WSDAN基本模型提高了1.6个百分点、0.34个百分点和0.12个百分点；仅利用VGG19网络的改进的模型的准确性和特异性相较WSDAN基本模型分别提高了0.7个百分点和2.82个百分点；以上两个方法结合使用的模型与WSDAN基本模型相比，准确性、敏感性和特异性分别提高了2.1个百分点、1.91个百分点和2.19个百分点。

关键词: 改进VGG网络, 弱监督, 细粒度分类, 数据增强, 阿尔兹海默症

CLC Number:

TP751

Shuang DENG, Xiaohai HE, Linbo QING, Honggang CHEN, Qizhi TENG. Weakly supervised fine-grained classification method of Alzheimer’s disease based on improved visual geometry group network[J]. Journal of Computer Applications, 2022, 42(1): 302-309.

邓爽, 何小海, 卿粼波, 陈洪刚, 滕奇志. 基于改进VGG网络的弱监督细粒度阿尔兹海默症分类方法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 302-309.

Figures/Tables 15

Fig. 1 WSDAN architecture

Fig. 2 Comparison of initial and enhanced MRI images of brain

Fig. 3 Structure of VGG19 network

Tab.1 VGG19 network parameters

卷积层	通道数	网络参数
Conv1	64	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool1	64	kernel： $2 × 2$ ，stride：2
Conv2	128	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool2	128	kernel： $2 × 2$ ，stride：2
Conv3	256	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool3	256	kernel： $2 × 2$ ，stride：2
Conv4	512	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool4	512	kernel： $2 × 2$ ，stride：2
Conv5	512	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool5	512	kernel： $2 × 2$ ，stride：2

Tab.1 VGG19 network parameters

卷积层	通道数	网络参数
Conv1	64	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool1	64	kernel： $2 × 2$ ，stride：2
Conv2	128	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool2	128	kernel： $2 × 2$ ，stride：2
Conv3	256	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool3	256	kernel： $2 × 2$ ，stride：2
Conv4	512	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool4	512	kernel： $2 × 2$ ，stride：2
Conv5	512	kernel： $3 × 3$ ，stride：1，padding：1
MaxPool5	512	kernel： $2 × 2$ ，stride：2

Fig. 4 Feature extraction part of VGG19 network

Fig. 5 Feature extraction part of improved VGG19 network

Fig. 6 Visualization comparison of feature networks

Fig. 7 Images in test set

Tab.2 Performance comparison of traditional classification networks

模型	准确性	敏感性	特异性
VGG19	92.20	92.49	91.87
ResNet101	91.90	92.81	90.93

Tab.3 WSDAN basic network models with different feature extraction networks

模型（特征网络）	准确性	敏感性	特异性
WSDAN（VGG19）	94.30	95.90	92.80
WSDAN（ResNet101）	95.10	97.40	92.81
WSDAN （Inception）	94.80	95.60	94.06

Tab.4 Comparison of training results of models with enhanced images and results of basic network models

模型（特征网络）	准确性	敏感性	特异性
WSDAN（VGG19）	94.30	95.90	92.80
WSDAN_d（VGG19）	95.90	96.24	93.12
WSDAN（ResNet101）	95.10	97.40	92.81
WSDAN_d（ResNet101）	95.40	96.24	93.10
WSDAN （Inception）	94.80	95.60	94.06
WSDAN_d （Inception）	95.90	95.93	95.92

Tab.5 Comparison of model with improved VGG19 network and model with basic VGG19 network

模型（特征网络）	准确性	敏感性	特异性
WSDAN（VGG19）	94.30	95.90	92.80
WSDAN（改进VGG19）	95.00	95.57	95.62

Tab.6 Comparison of adding different convolutional layers

模型（特征网络）	准确性	敏感性	特异性
WSDAN（改进_1）	92.60	93.12	92.18
WSDAN（改进_2）	95.40	93.75	97.19
WSDAN（改进_3）	96.40	97.81	94.99
WSDAN（改进_4）	87.03	92.49	81.56

Tab.7 Comparison of model with enhanced images combining improved network and basic network model

模型（特征网络）	准确性	敏感性	特异性
WSDAN（VGG19）	94.30	95.90	92.80
WSDAN_d（改进VGG19）	96.40	97.81	94.99

Tab. 8 Comparison of indicators of different classification networks

模型	准确性	敏感性	特异性
VGG19	92.20	92.49	91.87
ResNet101	91.90	92.81	90.93
NTS_Net	92.30	94.69	90.00
WSDAN	94.30	95.90	92.80
本文方法	96.40	97.81	94.99

References 23

1	叶玉如. 老年痴呆症——现代脑科学和医学研究面临的严峻挑战［J］. 生命科学， 2014， 26（1）：1. （YE Y R. Alzheimer’s disease： a tough challenge faced by modern brain science and medical research［J］. Chinese Bulletin of Life Science， 2014， 26（1）：1.）
2	BROOKMEYER R， JOHNSON E， ZIEGLER-GRAHAM K， et al. Forecasting the global burden of Alzheimer’s disease［J］. Alzheimer’s and Dementia， 2007， 3（3）：186-191. 10.1016/j.jalz.2007.04.381
3	林伟铭，高钦泉，杜民. 卷积神经网络诊断阿尔兹海默症的方法［J］.计算机应用， 2017， 37（12）：3504-3508. 10.11772/j.issn.1001-9081.2017.12.3504
	LIN W M， GAO Q Q， DU M. Convolutional neural network based method for the diagnosis of Alzheimer’s disease［J］. Journal of Computer Applications， 2017， 37（12）：3504-3508. 10.11772/j.issn.1001-9081.2017.12.3504
4	YANG Z， LUO T G， WANG D， et al. Learning to navigate for fine-grained classification［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11218. Chan： Springer， 2018：420-435.
5	LAM M， MAHASSENI B， TODOROVIC S. Fine-grained recognition as HSnet search for informative image parts［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：6497-6506. 10.1109/cvpr.2017.688
6	边小勇，江沛龄，赵敏，等. 基于多分支神经网络模型的弱监督细粒度图像分类方法［J］. 计算机应用， 2020， 40（5）：1295-1300.
	BIAN X Y， JIANG P L， ZHAO M， et al. Multi-branch neural network model based weakly supervised fine-grained image classification method［J］. Journal of Computer Applications， 2020， 40（5）：1295-1300.
7	陆鑫伟，余鹏飞，李海燕，等. 基于注意力自身线性融合的弱监督细粒度图像分类算法［J］. 计算机应用， 2021， 41（5）： 1319-1325. 10.1109/iaeac50856.2021.9390994
	LU X W， YU P F， LI H Y， et al. Weakly supervised fine-grained image classification algorithm based on attention-attention bilinear pooling［J］. Journal of Computer Applications ， 2021， 41（5）： 1319-1325. 10.1109/iaeac50856.2021.9390994
8	XIAO T J， XU Y C， YANG K Y， et al. The application of two-level attention models in deep convolutional neural network for fine-grained image classification［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015：842-850. 10.1109/cvpr.2015.7298685
9	JADERBERG M， SIMONYAN K， ZISSERMAN A， et al. Spatial transformer networks［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014： 2672-2680. 10.5244/c.28.88
10	FU J L， ZHENG H L， MEI T， et al. Look closer to see better： recurrent attention convolutional neural network for fine-grained image recognition［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：4476-4484. 10.1109/cvpr.2017.476
11	HU T， QI H G， HUANG Q M， et al. See better before looking closer： weakly supervised data augmentation network for fine-grained visual classification［EB/OL］. （2019-03-23）［2021-04-19］.. 10.1109/icme46284.2020.9102790
12	丁文谦，余鹏飞，李海燕，等. 基于Xception网络的弱监督细粒度图像分类［J/OL］. 计算机工程与应用. （2020-12-25）［2021-01-25］.，
	YU P F， LI H Y， et al. Weakly supervised fine-grained image classification based on Xception network［J/OL］. Computer Engineering and Applications. （2020-12-25）［2021-01-25］.
13	李振东，钟勇，陈蔓，等. 角度余量损失和中心损失联合的深度人脸识别［J］. 计算机应用， 2019， 39（S2）：55-58. 10.1109/icsidp47821.2019.9173230
	LI Z D， ZHONG Y， CHEN M， et al. Deep face recognition combined with angular margin loss and center loss［J］. Journal of Computer Applications， 2019， 39（S2）：55-58. 10.1109/icsidp47821.2019.9173230
14	朱学玲，刘丽. 图像增强中的平滑滤波技术［J］. 科技信息， 2012（32）：512. 10.3969/j.issn.1001-9960.2012.32.464
	ZHU X L， LIU L. Smooth filtering technology in image enhancement［J］. Science and Technology Information， 2012（32）：512. 10.3969/j.issn.1001-9960.2012.32.464
15	郭红伟，余江，朱家兴，等. 基于局部直方图的加权均值滤波器［J］. 计算机应用， 2010， 30（11）：3019-3021. 10.3724/sp.j.1087.2010.03019
	GUO H W， YU J， ZHU J X， et al. Weighted mean filter based on local histogram［J］. Journal of Computer Applications， 2010， 30（11）：3019-3021. 10.3724/sp.j.1087.2010.03019
16	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［EB/OL］. （2015-04-10）［2021-01-19］.. 10.5244/c.28.6
17	颜晨欢，李白，章超杰，等. 一种基于VGG的手写字母识别算法［J］. 信息技术与信息化， 2020（12）：63-65. 10.3969/j.issn.1672-9528.2020.12.019
	YAN C H， LI B， ZHANG C J， et al. A handwritten letter recognition algorithm based on VGG［J］. Information Technology and Informatization， 2020（12）：63-65. 10.3969/j.issn.1672-9528.2020.12.019
18	王羽徵，程远，毕海，等. 基于深度学习VGG网络的海洋单细胞藻类识别算法研究［J］. 大连海洋大学学报， 2021， 36（2）：334-339.
	WANG Y Z， CHENG Y， BI H， et al. Recognition algorithm of marine single-cell algae based on deep learning VGG network［J］. Journal of Dalian Ocean University， 2021， 36（2）：334-339.
19	陈津徽，张元良，尹泽睿. 基于改进的VGG19网络的面部表情识别［J］. 电脑知识与技术， 2020， 16（29）：187-188. 10.1117/12.2574468
	CHEN J H， ZHANG Y L， YIN Z R. Facial expression recognition based on improved VGG19 network［J］. Computer Knowledge and Technology， 2020， 16（29）：187-188. 10.1117/12.2574468
20	SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-CAM： visual explanations from deep networks via gradient-based localization［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017：618-626. 10.1109/iccv.2017.74
21	CHATTOPADHAY A， SARKAR A， HOWLADER P， et al. Grad-CAM++： generalized gradient-based visual explanations for deep convolutional networks［C］// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2018：839-847. 10.1109/wacv.2018.00097
22	陆小玲，吴海锋，曾玉，等. 3D迁移网络的阿尔茨海默症分类研究［J］. 计算机工程与应用， 2021， 57（16）：253-262. 10.1109/icccr49711.2021.9349393
	LU X L， WU H F， ZENG Y， et al. 3D transfer learning network for classification of Alzheimer’s disease［J］. Computer Engineering and Applications， 2021， 57（16）：253-262. 10.1109/icccr49711.2021.9349393
23	张柏雯，林岚，吴水才. 基于AlexNet模型的AD分类［J］. 北京工业大学学报， 2020， 46（1）：68-74. 10.11936/bjutxb2018070029
	ZHANG B W， LIN L， WU S C. Efficient Alzheimer’s disease classification based on AlexNet model［J］. Journal of Beijing University of Technology， 2020， 46（1）：68-74. 10.11936/bjutxb2018070029

[1]	Zimeng ZHU, Zhixin LI, Zhan HUAN, Ying CHEN, Jiuzhen LIANG. Weakly supervised video anomaly detection based on triplet-centered guidance [J]. Journal of Computer Applications, 2024, 44(5): 1452-1457.
[2]	Jiong WANG, Taotao TANG, Caiyan JIA. PAGCL： positive augmentation graph contrastive learning recommendation method without negative sampling [J]. Journal of Computer Applications, 2024, 44(5): 1485-1492.
[3]	Jie GUO, Jiayu LIN, Zuhong LIANG, Xiaobo LUO, Haitao SUN. Recommendation method based on knowledge‑awareness and cross-level contrastive learning [J]. Journal of Computer Applications, 2024, 44(4): 1121-1127.
[4]	Andi GUO, Zhen JIA, Tianrui LI. High-precision entity and relation extraction in medical domain based on pseudo-entity data augmentation [J]. Journal of Computer Applications, 2024, 44(2): 393-402.
[5]	Yifei SONG, Yi LIU. Fast adversarial training method based on data augmentation and label noise [J]. Journal of Computer Applications, 2024, 44(12): 3798-3807.
[6]	Xinrong HU, Jingxue CHEN, Zijian HUANG, Bangchao WANG, Xun YAO, Junping LIU, Qiang ZHU, Jie YANG. Graph convolution network-based masked data augmentation [J]. Journal of Computer Applications, 2024, 44(11): 3335-3344.
[7]	Qiang WANG, Xiaoming HUANG, Qiang TONG, Xiulei LIU. Weakly supervised salient object detection algorithm based on bounding box annotation [J]. Journal of Computer Applications, 2023, 43(6): 1910-1918.
[8]	Yimin CAO, Lei CAI, Jingyang GAO. Gene data generation method based on generative adversarial network [J]. Journal of Computer Applications, 2022, 42(3): 783-790.
[9]	Qiujie SUN, Jinggui LIANG, Si LI. Chinese grammatical error correction model based on bidirectional and auto-regressive transformers noiser [J]. Journal of Computer Applications, 2022, 42(3): 860-866.
[10]	Cong HU, Gang HUA. Weakly supervised action localization method based on attention mechanism [J]. Journal of Computer Applications, 2022, 42(3): 960-967.
[11]	Yu PENG, Yaolian SONG, Jun YANG. Motor imagery electroencephalography classification based on data augmentation [J]. Journal of Computer Applications, 2022, 42(11): 3625-3632.
[12]	Ping LUO, Ling DING, Xue YANG, Yang XIANG. Chinese event detection based on data augmentation and weakly supervised adversarial training [J]. Journal of Computer Applications, 2022, 42(10): 2990-2995.
[13]	Hongfei JIA, Xi LIU, Yu WANG, Hongbing XIAO, Suxia XING. Application of 3DPCANet in image classification of functional magnetic resonance imaging for Alzheimer’s disease [J]. Journal of Computer Applications, 2022, 42(1): 310-315.
[14]	JIA Chengxun, LAI Hua, YU Zhengtao, WEN Yonghua, YU Zhiqiang. Chinese-Vietnamese pseudo-parallel corpus generation based on monolingual language model [J]. Journal of Computer Applications, 2021, 41(6): 1652-1658.
[15]	GAN Lan, SHEN Hongfei, WANG Yao, ZHANG Yuejin. Data augmentation method based on improved deep convolutional generative adversarial networks [J]. Journal of Computer Applications, 2021, 41(5): 1305-1313.

Weakly supervised fine-grained classification method of Alzheimer’s disease based on improved visual geometry group network

基于改进VGG网络的弱监督细粒度阿尔兹海默症分类方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 23

Related Articles 15

Recommended Articles

Metrics