Auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation

doi:10.11772/j.issn.1001-9081.2024060856

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (5): 1410-1414.DOI: 10.11772/j.issn.1001-9081.2024060856

• China Conference on Data Mining 2024 (CCDM 2024) • Previous Articles

Auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation

Sijie NIU¹^,²(), Yuliang LIU¹^,²

^1.School of Information Science and Engineering，University of Jinan，Jinan Shandong 250022，China
^2.Shandong Provincial Key Laboratory of Network-based Intelligent Computing （University of Jinan），Jinan Shandong 250022，China

Received:2024-06-08 Revised:2024-09-05 Accepted:2024-09-10 Online:2024-10-08 Published:2025-05-10
Contact: Sijie NIU
About author:NIU Sijie， born in 1984， Ph. D.， professor. His research interests include pattern recognition， medical image analysis.
LIU Yuliang， born in 1999， M. S. Her research interests include medical image classification.
Supported by:
National Natural Science Foundation of China(62471202);Development Program of Youth Innovation Team of Institutions of Higher Learning in Shandong Province （Lujiaokehan ［2021］ No. 51）;Shandong Province Small- and Medium-sized Science and Technology Enterprises Innovation Capability Building Engineering Project(2022TSGC1048)

基于知识蒸馏双分支结构的视网膜病变辅助诊断方法

牛四杰¹^,²(), 刘昱良¹^,²

^1.济南大学信息科学与工程学院，济南 250022
^2.山东省网络环境智能计算技术重点实验室（济南大学），济南 250022

通讯作者: 牛四杰
作者简介:牛四杰（1984—），男，山东临沂人，教授，博士，CCF会员，主要研究方向：模式识别、医学影像分析
刘昱良（1999—），女，吉林长春人，硕士，主要研究方向：医学图像分类。
基金资助:
国家自然科学基金资助项目(62471202);山东省高等学校人才引育创新团队发展计划项目(鲁教科函［2021］51号);山东省科技型中小企业创新能力提升工程项目(2022TSGC1048)

Abstract

Abstract:

When using traditional models for the early diagnosis of retinopathy in high-risk patients with Diabetic Nephropathy （DN）， the diagnostic accuracy is often compromised due to limited and category imbalanced retinal images of diabetic patients. To address this issue， an auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation was proposed to improve the recognition capability for minority categories. Firstly， a teacher network pre-trained on large medical datasets was employed to guide the student network's learning process， transferring acquired knowledge to improve the student network's generalization ability and mitigate data scarcity. Secondly， a dual-branch structure was proposed in the student network. Branch 1 utilized a rebalancing strategy with Focal Loss function to emphasize challenging samples by adjusting loss function weights， while Branch 2 employed a Category Attention Module （CAM） to learn discriminative features for each category， preventing model bias towards majority categories. These two branches respectively promoted classifier learning and feature learning to alleviate category imbalance. Evaluated on clinically collected retinal image data， experimental results demonstrate that the proposed method achieves 1.05 and 1.53 percentage points improvements in accuracy and specificity respectively compared with Lesion-aware Attention Model （LAM） in screening tasks involving 66 cases （89 eyes） of high-risk patients with DN. The proposed method improves the recognition accuracy of DN and realizes the auxiliary diagnosis of retinal diseases.

Key words: deep learning, knowledge distillation, category imbalance, retinopathy image classification

摘要：

利用传统模型对糖尿病肾病（DN）高风险患者的视网膜疾病进行早期诊断时，由于糖尿病患者的视网膜图像数据少且类别不平衡，诊断精度不高。因此，提出一种基于知识蒸馏双分支结构的视网膜病变辅助诊断方法，以提高对少数类别的识别能力。该方法首先使用在大型医学数据集上训练的教师网络指导学生网络学习，将教师网络所学得的信息传递给学生网络，以提升学生网络的泛化能力，缓解数据少的问题。其次，在学生网络中提出一种双分支结构：分支一使用重平衡策略，引入Focal Loss函数，通过调节损失函数的权重使模型更关注难分样本；分支二利用类别注意力模块（CAM）学习每个类别的判别性特征，使模型在训练中不会偏向数据多的类别。这2个分支分别促进分类器学习和特征学习，可缓解类别不平衡。使用临床上收集的视网膜图像数据对所提方法进行评估，实验结果表明，所提方法在66例（89眼） DN高风险患者筛查任务上的准确率和特异度比病变感知注意力模型（LAM）分别提高了1.05和1.53个百分点。所提方法可以提高DN识别精度，实现视网膜疾病的辅助诊断。

关键词: 深度学习, 知识蒸馏, 类别不平衡, 视网膜病变图像分类

CLC Number:

TP391.3

Sijie NIU, Yuliang LIU. Auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation[J]. Journal of Computer Applications, 2025, 45(5): 1410-1414.

牛四杰, 刘昱良. 基于知识蒸馏双分支结构的视网膜病变辅助诊断方法[J]. 《计算机应用》唯一官方网站, 2025, 45(5): 1410-1414.

Figures/Tables 6

References 31

1	ROGLIC G. WHO global report on diabetes： a summary ［J］. International Journal of Noncommunicable Diseases， 2016， 1（1）： 3-8.
2	TINAJERO M G， MALIK V S. An update on the epidemiology of type 2 diabetes： a global perspective ［J］. Endocrinology and Metabolism Clinics， 2021， 50（3）： 337-355.
3	PAPATHEODOROU K， BANACH M， BEKIARI E， et al. Complications of diabetes 2017 ［J］. Journal of Diabetes Research， 2018， 2018： No.3086167.
4	TOMIC D， SHAW J E， MAGLIANO D J. The burden and risks of emerging complications of diabetes mellitus ［J］. Nature Reviews Endocrinology， 2022， 18（9）： 525-539.
5	GROSS J L， DE AZEVEDO M J， SILVEIRO S P， et al. Diabetic nephropathy： diagnosis， prevention， and treatment ［J］. Diabetes Care， 2005， 28（1）： 164-176.
6	KIKKAWA R， KOYA D， HANEDA M. Progression of diabetic nephropathy ［J］. American Journal of Kidney Diseases， 2003， 41（3S）： S19-S21.
7	SAGOO M K， GNUDI L. Diabetic nephropathy： an overview ［M］// Diabetic nephropathy： methods and protocols， MIMB 2067. New York： Humana， 2020： 3-7.
8	VUJOSEVIC S， ALDINGTON S J， SILVA P， et al. Screening for diabetic retinopathy： new perspectives and challenges ［J］. The Lancet Diabetes and Endocrinology， 2020， 8（4）： 337-347.
9	JAWA A， KCOMT J， FONSECA V A. Diabetic nephropathy and retinopathy ［J］. Medical Clinics， 2004， 88（4）： 1001-1036.
10	ABRÀMOFF M D， GARVIN M K， SONKA M. Retinal imaging and image analysis ［J］. IEEE Reviews in Biomedical Engineering， 2010， 3： 169-208.
11	ZHOU B， CUI Q， WEI X S， et al. BBN： bilateral-branch network with cumulative learning for long-tailed visual recognition ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 9719-9728.
12	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2999-3007.
13	XIE Y， WAN Q， XIE H， et al. Fundus image-label pairs synthesis and retinopathy screening via GANs with class-imbalanced semi-supervised learning ［J］. IEEE Transactions on Medical Imaging， 2023， 42（9）：2714-2725.
14	聂永琦，曹慧，杨锋，等.深度学习在糖尿病视网膜病灶检测中的应用综述［J］.计算机工程与应用，2021，57（20）：25-41.
	NIE Y Q， CAO H， YANG F， et al. Review of application of deep learning in detection of diabetic retinal lesions［J］. Computer Engineering and Applications， 2021， 57（20）： 25-41.
15	POPLIN R， VARADARAJAN A V， BLUMER K， et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning ［J］. Nature Biomedical Engineering， 2018， 2（3）： 158-164.
16	KANG E Y C， HSIEH Y T， LI C H， et al. Deep learning-based detection of early renal function impairment using retinal fundus images： model development and validation ［J］. JMIR Medical Informatics， 2020， 8（11）： No.e23472.
17	ZHAO L， REN H， ZHANG J， et al. Diabetic retinopathy， classified using the lesion-aware deep learning system， predicts diabetic end-stage renal disease in Chinese patients ［J］. Endocrine Practice， 2020， 26（4）： 429-443.
18	SABANAYAGAM C， XU D， TING D S W， et al. A deep learning algorithm to detect chronic kidney disease from retinal photographs in community-based populations ［J］. The Lancet Digital Health， 2020， 2（6）： e295-e302.
19	BETZLER B K， CHEE E Y L， HE F， et al. Deep learning algorithms to detect diabetic kidney disease from retinal photographs in multiethnic populations with diabetes ［J］. Journal of the American Medical Informatics Association， 2023， 30（12）： 1904-1914.
20	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network ［EB/OL］. ［2024-02-15］. .
21	CHEN S， MA K， ZHENG Y. Med 3D： transfer learning for 3D medical image analysis ［EB/OL］. ［2023-12-19］. .
22	CHATTERJEE S， KHUNTI K， DAVIES M J. Type 2 diabetes ［J］. The Lancet， 2017， 389（10085）： 2239-2251.
23	YANG Z， TAN T E， SHAO Y， et al. Classification of diabetic retinopathy： past， present and future ［J］. Frontiers in Endocrinology， 2022， 13： 1079217.
24	HANEDA M， UTSUNOMIYA K， KOYA D， et al. A new classification of diabetic nephropathy 2014： a report from joint committee on diabetic nephropathy［J］. Clinical and Experimental Nephrology， 2015， 19（1）： 1-5.
25	KINGMA D P， BA J L. Adam： a method for stochastic optimization ［EB/OL］. ［2024-01-09］. .
26	CUI Y， JIA M， LIN T Y， et al. Class-balanced loss based on effective number of samples ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9260-9269.
27	CAO K， WEI C， GAIDON A， et al. Learning imbalanced datasets with label-distribution-aware margin loss ［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2019： 1567-1578.
28	SADI A A， CHOWDHURY L， JAHAN N， et al. LMFLOSS： a hybrid loss for imbalanced medical image classification ［EB/OL］. ［2024-03-01］. .
29	LIU Y， ZHANG F， GAO X， et al. Lesion-aware attention network for diabetic nephropathy diagnosis with optical coherence tomography images［J］. Frontiers in Medicine， 2023， 10： No.1259478.
30	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision ［C］// Proceedings of the 2016 IEEE conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826.
31	TAN M， LE Q V. EfficientNet： rethinking model scaling for convolutional neural networks［C］// Proceedings of the 36th International Conference on Machine Learning. New York： ACM， 2019： 6105-6114.

方法	准确率	灵敏度	特异度
ResNet18	84.51	74.44	88.89
InceptionV3^［30］	83.35	81.11	84.49
EfficientNet^［31］	78.39	54.44	86.64
Knowledge Distillation	86.61	81.11	88.97
WeightedRandomSampler	86.61	77.78	90.43
Weighted Cross Entropy	85.56	81.11	87.43
Class Balance Loss^［26］	88.72	81.11	92.25
LDAM Loss^［27］	88.52	83.33	90.64
LMF Loss^［28］	88.72	84.44	90.51
LAM^［29］	91.68	89.99	92.18
本文方法	92.73	89.99	93.71

方法	准确率	灵敏度	特异度
ResNet18	84.51	74.44	88.89
InceptionV3^［30］	83.35	81.11	84.49
EfficientNet^［31］	78.39	54.44	86.64
Knowledge Distillation	86.61	81.11	88.97
WeightedRandomSampler	86.61	77.78	90.43
Weighted Cross Entropy	85.56	81.11	87.43
Class Balance Loss^［26］	88.72	81.11	92.25
LDAM Loss^［27］	88.52	83.33	90.64
LMF Loss^［28］	88.72	84.44	90.51
LAM^［29］	91.68	89.99	92.18
本文方法	92.73	89.99	93.71

Focal Loss	CAM	知识蒸馏	准确率	灵敏度	特异度
			84.51	74.44	88.89
√			86.61	84.44	87.43
	√		87.57	84.44	89.10
		√	86.61	81.11	88.97
√	√		89.57	87.77	90.76
√	√	√	92.73	89.99	93.71

Focal Loss	CAM	知识蒸馏	准确率	灵敏度	特异度
			84.51	74.44	88.89
√			86.61	84.44	87.43
	√		87.57	84.44	89.10
		√	86.61	81.11	88.97
√	√		89.57	87.77	90.76
√	√	√	92.73	89.99	93.71

损失函数	准确率	灵敏度	特异度
L_CE	87.66	83.33	88.84
L_WCE	87.66	86.66	87.31
L_FL	92.73	89.99	93.71

Auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation

基于知识蒸馏双分支结构的视网膜病变辅助诊断方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 31

Related Articles 15

Recommended Articles

Metrics

[1]	Lihu PAN, Shouxin PENG, Rui ZHANG, Zhiyang XUE, Xuzhen MAO. Video anomaly detection for moving foreground regions [J]. Journal of Computer Applications, 2025, 45(4): 1300-1309.
[2]	Yiding WANG, Zehao WANG, Yaoli LI, Shaoqing CAI, Yuan YUAN. Multi-scale 2D-Adaboost microscopic image recognition algorithm of Chinese medicinal materials powder [J]. Journal of Computer Applications, 2025, 45(4): 1325-1332.
[3]	Yang ZHOU, Hui LI. Remote sensing image building extraction network based on dual promotion of semantic and detailed features [J]. Journal of Computer Applications, 2025, 45(4): 1310-1316.
[4]	Ruilong CHEN, Tao HU, Youjun BU, Peng YI, Xianjun HU, Wei QIAO. Stacking ensemble adversarial defense method for encrypted malicious traffic detection model [J]. Journal of Computer Applications, 2025, 45(3): 864-871.
[5]	Zhenhua XUE, Qiang LI, Chao HUANG. Vision foundation model-driven pixel-level image anomaly detection method [J]. Journal of Computer Applications, 2025, 45(3): 823-831.
[6]	Zirong HONG, Guangqing BAO. Review of radar automatic target recognition based on ensemble learning [J]. Journal of Computer Applications, 2025, 45(2): 371-382.
[7]	Sheng YANG, Yan LI. Contrastive knowledge distillation method for object detection [J]. Journal of Computer Applications, 2025, 45(2): 354-361.
[8]	Zhongwei ZHANG, Jun WANG, Shudong LIU, Zhiheng WANG. Object detection in remote sensing image based on multi-scale feature fusion and weighted boxes fusion [J]. Journal of Computer Applications, 2025, 45(2): 633-639.
[9]	Tianqi ZHANG, Shuang TAN, Xiwen SHEN, Juan TANG. Image watermarking method combining attention mechanism and multi-scale feature [J]. Journal of Computer Applications, 2025, 45(2): 616-623.
[10]	Miaolei DENG, Yupei KAN, Chuanchuan SUN, Haihang XU, Shaojun FAN, Xin ZHOU. Summary of network intrusion detection systems based on deep learning [J]. Journal of Computer Applications, 2025, 45(2): 453-466.
[11]	Songsen YU, Zhifan LIN, Guopeng XUE, Jianyu XU. Lightweight large-format tile defect detection algorithm based on improved YOLOv8 [J]. Journal of Computer Applications, 2025, 45(2): 647-654.
[12]	Danni DING, Bo PENG, Xi WU. VPNet： fatty liver ultrasound image classification method inspired by ventral pathway [J]. Journal of Computer Applications, 2025, 45(2): 662-669.
[13]	Yan LI, Guanhua YE, Yawen LI, Meiyu LIANG. Enterprise ESG indicator prediction model based on richness coordination technology [J]. Journal of Computer Applications, 2025, 45(2): 670-676.
[14]	Siqi ZHANG, Jinjun ZHANG, Tianyi WANG, Xiaolin QIN. Deep temporal event detection algorithm based on signal temporal logic [J]. Journal of Computer Applications, 2025, 45(1): 90-97.
[15]	Zongsheng ZHENG, Jia DU, Yuhe CHENG, Zecheng ZHAO, Yuewei ZHANG, Xulong WANG. Cross-modal dual-stream alternating interactive network for infrared-visible image classification [J]. Journal of Computer Applications, 2025, 45(1): 275-283.