Semi-supervised knee abnormality classification based on multi-imaging center MRI data

doi:10.11772/j.issn.1001-9081.2021010200

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 316-324.DOI: 10.11772/j.issn.1001-9081.2021010200

• Frontier and comprehensive applications • Previous Articles Next Articles

Semi-supervised knee abnormality classification based on multi-imaging center MRI data

Jie WU¹^,², Shitian ZHANG³, Haibin XIE¹^,², Guang YANG¹^,²()

^1.School of Physics and Electronic Science，East China Normal University，Shanghai 200241，China
^2.Shanghai Key Laboratory of Magnetic Resonance （East China Normal University），Shanghai 200062，China
^3.School of Physics and Astronomy，University of Edinburgh，Edinburgh EH8 9YL，UK

Received:2021-02-03 Revised:2021-04-28 Accepted:2020-04-29 Online:2021-05-07 Published:2022-01-10
Contact: Guang YANG
About author:WU Jie， born in 1995， M. S. candidate. Her research interests include machine learning， medical image processing.
ZHANG Shitian， born in 1995， M. S. His research interests include machine learning， non-equilibrium statistical physics.
XIE Haibin， born in 1967， Ph. D.， engineer. His research interests include machine learning， medical image processing.
YANG Guang， born in 1968， Ph. D.， associate research fellow. His research interests include machine learning， medical image processing.
Supported by:
National Natural Science Foundation of China(61731009)

基于多影像中心磁共振成像数据的半监督膝盖异常分类

吴洁¹^,², 张师天³, 谢海滨¹^,², 杨光¹^,²()

^1.华东师范大学物理与电子科学学院, 上海 200241
^2.上海市磁共振重点实验室(华东师范大学), 上海 200062
^3.爱丁堡大学物理与天文学院, 爱丁堡 EH8 9YL, 英国

通讯作者: 杨光
作者简介:吴洁（1995—），女，江苏扬州人，硕士研究生，主要研究方向：机器学习、医学图像处理
张师天（1995—），男，江苏扬州人，硕士，主要研究方向：机器学习、非平衡态统计物理
谢海滨（1967—），男，江苏泗洪人，工程师，博士，主要研究方向：机器学习、医学图像处理
杨光（1968—），男，安徽蚌埠人，副研究员，博士，主要研究方向：机器学习、医学图像处理。
基金资助:
国家自然科学基金资助项目(61731009)

Abstract

Abstract:

The manual labeling of abundant data is laborious and the amount of Magnetic Resonance Imaging （MRI） data from a single imaging center is limited. Concerning the above problems， a Magnetic Resonance Semi-Supervised Learning （MRSSL） method utilizing multi-imaging center labeled and unlabeled MRI data was proposed and applied to knee abnormality classification. Firstly， data augmentation was used to provide the inductive bias required by the model . Next， the classification loss and the consistency loss were combined to constraint an artificial neural network to extract the discriminative features from the data. Then， the features were used for the MRI knee abnormality classification. Additionally， the corresponding Magnetic Resonance Supervised Learning （MRSL） method only using labeled samples was proposed and compared with MRSSL for the same labeled samples. The results demonstrate that MRSSL surpasses MRSL in both model classification performance and model generalization ability. Finally， MRSSL was compared with other semi-supervised learning methods. The results indicate that data augmentation plays an important role on performance improvement， and with stronger inclusiveness for MRI data， MRSSL outperforms others on the knee abnormality classification.

Key words: deep learning, semi-supervised learning, Magnetic Resonance Imaging (MRI), multi-imaging center data, knee abnormality classification, MRNet dataset, OAI （OsteoArthritis Initiative） dataset

摘要：

针对大量数据手工标记的繁重性和单一影像中心磁共振成像（MRI）数据的有限性问题，提出了一种利用多影像中心有标签与无标签MRI数据的用于磁共振的半监督学习（MRSSL）方法，并将其应用在膝盖异常分类任务中。首先，运用了数据扩增方法来提供模型所需的归纳偏置；接着，融合了分类损失项和一致性损失项来约束人工神经网络并使之从数据中提取出具有辨别力的特征；然后，将这些特征用于MRI膝盖异常分类。此外，也提出了对应的仅利用有标签数据的完全监督学习（MRSL）方法。在给出同样的有标签样本时，将MRSL与MRSSL进行了比较，结果表明MRSSL的模型分类性能与泛化性能明显优于MRSL。最后，将MRSSL与其他半监督学习方法进行了比较。结果表明数据扩增在性能提升中起到了重要作用，并且MRSSL凭借更强的MRI数据包容性取得了最优的膝盖异常分类性能。

关键词: 深度学习, 半监督学习, 磁共振成像, 多影像中心数据, 膝盖异常分类, MRNet数据集, OAI数据集

CLC Number:

TP391.41

Jie WU, Shitian ZHANG, Haibin XIE, Guang YANG. Semi-supervised knee abnormality classification based on multi-imaging center MRI data[J]. Journal of Computer Applications, 2022, 42(1): 316-324.

吴洁, 张师天, 谢海滨, 杨光. 基于多影像中心磁共振成像数据的半监督膝盖异常分类[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 316-324.

Figures/Tables 9

Tab. 1 Distribution of original training set and validation set of MRNet dataset and training set and validation set used in the paper

数据集	膝盖正常	膝盖异常				合计
数据集	膝盖正常	前交叉韧带撕裂	半月板撕裂	前交叉韧带与半月板都撕裂	其他膝盖异常	合计
MRNet训练集	217	83	272	125	433	1 130
MRNet验证集	25	23	21	31	20	120
本文训练集	193	59	248	101	409	1 010
本文验证集	24	24	24	24	24	120

Fig. 1 Structure of MRSL and MRSSL

Tab. 2 Five augmentation levels of MRAugment

等级	颜色变换		仿射变换
等级	亮度 $β$	对比度 $γ$	水平翻转 $φ$	缩放 $σ$	旋转 $ρ$	平移 $τ$
0	0.0	0.0	0.5	0.0	0	0.10
1	0.1	0.1	0.5	0.3	15	0.05
2	0.2	0.2	0.5	0.3	25	0.10
3	0.3	0.3	0.5	0.3	35	0.15
4	0.4	0.4	0.5	0.3	45	0.20

Tab. 2 Five augmentation levels of MRAugment

等级	颜色变换		仿射变换
等级	亮度 $β$	对比度 $γ$	水平翻转 $φ$	缩放 $σ$	旋转 $ρ$	平移 $τ$
0	0.0	0.0	0.5	0.0	0	0.10
1	0.1	0.1	0.5	0.3	15	0.05
2	0.2	0.2	0.5	0.3	25	0.10
3	0.3	0.3	0.5	0.3	35	0.15
4	0.4	0.4	0.5	0.3	45	0.20

Fig. 2 Architecture of network

Tab. 3 AUC statistics of MRSL with different MRAugment augmentation levels and MRSSL with different MRAugment level combinations

方法	MRAugment等级/等级组合	验证集AUC			测试集AUC
方法	MRAugment等级/等级组合	平均值	标准差	中位数	平均值	标准差	中位数
MRSL	$α = 0$	0.936	0.008	0.938	0.856	0.012	0.858
	$α = 1$	0.956	0.010	0.960	0.916	0.012	0.914
	$α = 2$	0.965	0.009	0.965	0.919	0.013	0.915
	$α = 3$	0.961	0.013	0.964	0.906	0.013	0.909
MRSSL	$α, A, a = 1,1, 0$	0.947	0.010	0.946	0.928	0.012	0.926
	$α, A, a = 1,2, 0$	0.954	0.009	0.957	0.939	0.006	0.939
	$α, A, a = 1,3, 0$	0.960	0.008	0.960	0.946	0.010	0.949
	$α, A, a = 1,4, 0$	0.951	0.005	0.952	0.944	0.015	0.944
	$α, A, a = 2,2, 0$	0.951	0.011	0.950	0.942	0.009	0.945
	$α, A, a = 2,3, 0$	0.951	0.011	0.952	0.942	0.008	0.941
	$α, A, a = 2,4, 0$	0.949	0.007	0.947	0.933	0.010	0.931

Tab. 3 AUC statistics of MRSL with different MRAugment augmentation levels and MRSSL with different MRAugment level combinations

方法	MRAugment等级/等级组合	验证集AUC			测试集AUC
方法	MRAugment等级/等级组合	平均值	标准差	中位数	平均值	标准差	中位数
MRSL	$α = 0$	0.936	0.008	0.938	0.856	0.012	0.858
	$α = 1$	0.956	0.010	0.960	0.916	0.012	0.914
	$α = 2$	0.965	0.009	0.965	0.919	0.013	0.915
	$α = 3$	0.961	0.013	0.964	0.906	0.013	0.909
MRSSL	$α, A, a = 1,1, 0$	0.947	0.010	0.946	0.928	0.012	0.926
	$α, A, a = 1,2, 0$	0.954	0.009	0.957	0.939	0.006	0.939
	$α, A, a = 1,3, 0$	0.960	0.008	0.960	0.946	0.010	0.949
	$α, A, a = 1,4, 0$	0.951	0.005	0.952	0.944	0.015	0.944
	$α, A, a = 2,2, 0$	0.951	0.011	0.950	0.942	0.009	0.945
	$α, A, a = 2,3, 0$	0.951	0.011	0.952	0.942	0.008	0.941
	$α, A, a = 2,4, 0$	0.949	0.007	0.947	0.933	0.010	0.931

Fig. 3 AUC changes of MRSL with different MRAugment augmentation levels and MRSSL with different MRAugment level combinations

Fig. 4 Relationship between AUC and maximum value of consistency weight coefficient in MRSSL

Tab. 4 Result comparison between the selected models of MRSL and MRSSL on 5 evaluation indicators

方法	AUC	准确率	敏感性	特异性	G-mean
两者差值	0.043	0.025	-0.009	0.160	0.085
MRSL	0.908 ［0.838，0.978］	0.858 ［0.785，0.910］	0.916 ［0.843，0.957］	0.640 ［0.445，0.798］	0.766
MRSSL	0.951 ［0.913，0.989］	0.883 ［0.814，0.929］	0.905 ［0.830，0.949］	0.800 ［0.609，0.911］	0.851

Tab. 5 Result comparison of different methods

方法	AUC	准确率	敏感性	特异性	G-mean
MRSL（数据扩增）	0.919±0.013	0.876±0.009	0.918±0.005	0.716±0.047	0.810±0.026
MRSL（高斯噪声）	0.887±0.019	0.849±0.022	0.905±0.029	0.636±0.081	0.757±0.046
PI Model	0.890±0.023	0.844±0.018	0.882±0.034	0.698±0.133	0.780±0.062
FixMatch	0.930±0.011	0.875±0.021	0.901±0.035	0.778±0.078	0.835±0.034
UDA	0.937±0.014	0.882±0.016	0.916±0.024	0.756±0.061	0.831±0.030
MRSSL	0.946±0.010	0.893±0.013	0.925±0.019	0.769±0.052	0.843±0.025

References 30

1	LITJENS G， KOOI T， EHTESHAMI BEJNORDI B， et al. A survey on deep learning in medical image analysis［J］. Medical Image Analysis， 2017， 42：60-88. 10.1016/j.media.2017.07.005
2	SHEN D G， WU G R， SUK H I. Deep learning in medical image analysis［J］. Annual Review of Biomedical Engineering， 2017， 19： 221-248. 10.1146/annurev-bioeng-071516-044442
3	SONG Y， ZHANG Y D， YAN X， et al. Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI［J］. Journal of Magnetic Resonance Imaging， 2018， 48（6）： 1570-1577. 10.1002/jmri.26047
4	HERENT P， SCHMAUCH B， JEHANNO P， et al. Detection and characterization of MRI breast lesions using deep learning［J］. Diagnostic and Interventional Imaging， 2019， 100（4）： 219-225. 10.1016/j.diii.2019.02.008
5	ZHOU Z Y， ZHAO G Y， KIJOWSKI R， et al. Deep convolutional neural network for segmentation of knee joint anatomy［J］. Magnetic Resonance in Medicine， 2018， 80（6）： 2759-2770. 10.1002/mrm.27229
6	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
7	HE K M， ZHANG X Y， REN S Q， et al. Identity mappings in deep residual networks［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS9908. Cham： Springer， 2016： 630-645.
8	ZAGORUYKO S， KOMODAKIS N. Wide residual networks［C］// Proceedings of the 2016 British Machine Vision Conference. Durham： BMVA Press， 2016： No.87. 10.5244/c.30.87
9	PAN S J， YANG Q. A survey on transfer learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2010， 22（10）： 1345-1359. 10.1109/tkde.2009.191
10	BIEN N， RAJPURKAR P， BALL RL， et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging： development and retrospective validation of MRNet［J］. PLoS Medicine， 2018， 15（11）： No.e1002699. 10.1371/journal.pmed.1002699
11	DENG J， DONG W， SOCHER R， et al. ImageNet： a large-scale hierarchical image database［C］// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2009： 248-255. 10.1109/cvpr.2009.5206848
12	ZHU X J， GOLDBERG A B. Introduction to Semi-Supervised Learning［M］. San Rafael， CA： Morgan & Claypool Publishers， 2009： 23-33. 10.3115/1621829.1621832
13	LAINE S， AILA T. Temporal ensembling for semi-supervised learning［EB/OL］. （2017-03-15）［2020-12-12］..
14	MIYATO T， MAEDA S I， KOYAMA M， et al. Virtual adversarial training： a regularization method for supervised and semi-supervised learning［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2019， 41（8）： 1979-1993. 10.1109/tpami.2018.2858821
15	TARVAINEN A， VALPOLA H. Mean teachers are better role models： weight-averaged consistency targets improve semi-supervised deep learning results［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates， Inc.， 2017： 1195-1204.
16	XIE Q Z， DAI Z H， HOVY E， et al. Unsupervised data augmentation for consistency training［C/OL］// Proceedings of the 34th International Conference on Neural Information Processing Systems. ［2020-12-12］..
17	CHEPLYGINA V， DE BRUIJNE M， PLUIM J P W. Not-so-supervised： a survey of semi-supervised， multi-instance， and transfer learning in medical image analysis［J］. Medical Image Analysis， 2019， 54： 280-296. 10.1016/j.media.2019.03.009
18	LIU Q D， YU L Q， LUO L Y， et al. Semi-supervised medical image classification with relation-driven self-ensembling model［J］. IEEE Transactions on Medical Imaging， 2020， 39（11）： 3429-3440. 10.1109/tmi.2020.2995518
19	NGUYEN H H， SAARAKKALA S， BLASCHKO M B， et al. Semixup： in- and out-of-manifold regularization for deep semi-supervised knee osteoarthritis severity grading from plain radiographs［J］. IEEE Transactions on Medical Imaging， 2020， 39： 4346-4356. 10.1109/tmi.2020.3017007
20	PETERFY C G， SCHNEIDER E， NEVITT M. The osteoarthritis initiative： report on the design rationale for the magnetic resonance imaging protocol for the knee［J］. Osteoarthritis and Cartilage， 2008， 16（12）： 1433-1441. 10.1016/j.joca.2008.06.016
21	HU J L， KUANG Y Z， LIAO B， et al. A multichannel 2D convolutional neural network model for task-evoked fMRI data classification［J］. Computational Intelligence and Neuroscience， 2019， 2019： No.5065214. 10.1155/2019/5065214
22	MITCHELL T M. The need for biases in learning generalizations： Rutgers CS tech report CBM-TR-117［R］. New Brunswick， NJ： Rutgers University， 1980： 1-3.
23	CUBUK E D， ZOPH B， SHLENS J， et al. RandAugment： practical automated data augmentation with a reduced search space［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2020： 3008-3017. 10.1109/cvprw50498.2020.00359
24	NYÚL L G， UDUPA J K. On standardizing the MR image intensity scale［J］. Magnetic Resonance in Medicine， 1999， 42（6）： 1072-1081. 10.1002/(sici)1522-2594(199912)42:6<1072::aid-mrm11>3.0.co;2-m
25	LOWEKAMP B C， CHEN D T， IBÁÑEZ L， et al. The design of simpleITK［J］. Frontiers in Neuroinformatics， 2013， 7： No.45. 10.3389/fninf.2013.00045
26	YANIV Z， LOWEKAMP B C， JOHNSON H J， et al. SimpleITK image-analysis notebooks： a collaborative environment for education and reproducible research［J］. Journal of Digital Imaging， 2018， 31（3）： 290-303. 10.1007/s10278-017-0037-8
27	WALT S VAN DER， SCHÖNBERGER J L， NUNEZ-IGLESIAS J， et al. Scikit-image： image processing in Python［J］. PeerJ， 2014， 2： No.e453. 10.7717/peerj.453
28	JAPKOWICZ N， STEPHEN S. The class imbalance problem： a systematic study［J］. Intelligent Data Analysis， 2002， 6（5）： 429-449. 10.3233/ida-2002-6504
29	SOHN K， BERTHELOT D， LI C L， et al. FixMatch： simplifying semi-supervised learning with consistency and confidence［C/OL］// Proceedings of the 34th International Conference on Neural Information Processing Systems. ［2020-12-12］..
30	PASZKE A， GROSS S， MASSA F， et al. PyTorch： an imperative style， high-performance deep learning library［C/OL］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. ［2020-12-12］..

[1]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[2]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[3]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[4]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[5]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[6]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[7]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[8]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[9]	Yingjun ZHANG, Niuniu LI, Binhong XIE, Rui ZHANG, Wangdong LU. Semi-supervised object detection framework guided by curriculum learning [J]. Journal of Computer Applications, 2024, 44(8): 2326-2333.
[10]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[11]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[12]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[13]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[14]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[15]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.

Semi-supervised knee abnormality classification based on multi-imaging center MRI data

基于多影像中心磁共振成像数据的半监督膝盖异常分类

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 30

Related Articles 15

Recommended Articles

Metrics