《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (1): 316-324.DOI: 10.11772/j.issn.1001-9081.2021010200
• 前沿与综合应用 • 上一篇
收稿日期:
2021-02-03
修回日期:
2021-04-28
接受日期:
2020-04-29
发布日期:
2021-05-07
出版日期:
2022-01-10
通讯作者:
杨光
作者简介:
吴洁(1995—),女,江苏扬州人,硕士研究生,主要研究方向:机器学习、医学图像处理基金资助:
Jie WU1,2, Shitian ZHANG3, Haibin XIE1,2, Guang YANG1,2()
Received:
2021-02-03
Revised:
2021-04-28
Accepted:
2020-04-29
Online:
2021-05-07
Published:
2022-01-10
Contact:
Guang YANG
About author:
WU Jie, born in 1995, M. S. candidate. Her research interests include machine learning, medical image processing.Supported by:
摘要:
针对大量数据手工标记的繁重性和单一影像中心磁共振成像(MRI)数据的有限性问题,提出了一种利用多影像中心有标签与无标签MRI数据的用于磁共振的半监督学习(MRSSL)方法,并将其应用在膝盖异常分类任务中。首先,运用了数据扩增方法来提供模型所需的归纳偏置;接着,融合了分类损失项和一致性损失项来约束人工神经网络并使之从数据中提取出具有辨别力的特征;然后,将这些特征用于MRI膝盖异常分类。此外,也提出了对应的仅利用有标签数据的完全监督学习(MRSL)方法。在给出同样的有标签样本时,将MRSL与MRSSL进行了比较,结果表明MRSSL的模型分类性能与泛化性能明显优于MRSL。最后,将MRSSL与其他半监督学习方法进行了比较。结果表明数据扩增在性能提升中起到了重要作用,并且MRSSL凭借更强的MRI数据包容性取得了最优的膝盖异常分类性能。
中图分类号:
吴洁, 张师天, 谢海滨, 杨光. 基于多影像中心磁共振成像数据的半监督膝盖异常分类[J]. 计算机应用, 2022, 42(1): 316-324.
Jie WU, Shitian ZHANG, Haibin XIE, Guang YANG. Semi-supervised knee abnormality classification based on multi-imaging center MRI data[J]. Journal of Computer Applications, 2022, 42(1): 316-324.
数据集 | 膝盖正常 | 膝盖异常 | 合计 | |||
---|---|---|---|---|---|---|
前交叉韧带撕裂 | 半月板撕裂 | 前交叉韧带与半月板都撕裂 | 其他膝盖异常 | |||
MRNet训练集 | 217 | 83 | 272 | 125 | 433 | 1 130 |
MRNet验证集 | 25 | 23 | 21 | 31 | 20 | 120 |
本文训练集 | 193 | 59 | 248 | 101 | 409 | 1 010 |
本文验证集 | 24 | 24 | 24 | 24 | 24 | 120 |
表1 MRNet数据集的原始训练集、验证集与本文实验中训练集、验证集的膝盖情况分布
Tab. 1 Distribution of original training set and validation set of MRNet dataset and training set and validation set used in the paper
数据集 | 膝盖正常 | 膝盖异常 | 合计 | |||
---|---|---|---|---|---|---|
前交叉韧带撕裂 | 半月板撕裂 | 前交叉韧带与半月板都撕裂 | 其他膝盖异常 | |||
MRNet训练集 | 217 | 83 | 272 | 125 | 433 | 1 130 |
MRNet验证集 | 25 | 23 | 21 | 31 | 20 | 120 |
本文训练集 | 193 | 59 | 248 | 101 | 409 | 1 010 |
本文验证集 | 24 | 24 | 24 | 24 | 24 | 120 |
等级 | 颜色变换 | 仿射变换 | ||||
---|---|---|---|---|---|---|
亮度 | 对比度 | 水平翻转 | 缩放 | 旋转 | 平移 | |
0 | 0.0 | 0.0 | 0.5 | 0.0 | 0 | 0.10 |
1 | 0.1 | 0.1 | 0.5 | 0.3 | 15 | 0.05 |
2 | 0.2 | 0.2 | 0.5 | 0.3 | 25 | 0.10 |
3 | 0.3 | 0.3 | 0.5 | 0.3 | 35 | 0.15 |
4 | 0.4 | 0.4 | 0.5 | 0.3 | 45 | 0.20 |
表2 MRAugment的5种扩增等级
Tab. 2 Five augmentation levels of MRAugment
等级 | 颜色变换 | 仿射变换 | ||||
---|---|---|---|---|---|---|
亮度 | 对比度 | 水平翻转 | 缩放 | 旋转 | 平移 | |
0 | 0.0 | 0.0 | 0.5 | 0.0 | 0 | 0.10 |
1 | 0.1 | 0.1 | 0.5 | 0.3 | 15 | 0.05 |
2 | 0.2 | 0.2 | 0.5 | 0.3 | 25 | 0.10 |
3 | 0.3 | 0.3 | 0.5 | 0.3 | 35 | 0.15 |
4 | 0.4 | 0.4 | 0.5 | 0.3 | 45 | 0.20 |
方法 | MRAugment等级/等级组合 | 验证集AUC | 测试集AUC | ||||
---|---|---|---|---|---|---|---|
平均值 | 标准差 | 中位数 | 平均值 | 标准差 | 中位数 | ||
MRSL | 0.936 | 0.008 | 0.938 | 0.856 | 0.012 | 0.858 | |
0.956 | 0.010 | 0.960 | 0.916 | 0.012 | 0.914 | ||
0.965 | 0.009 | 0.965 | 0.919 | 0.013 | 0.915 | ||
0.961 | 0.013 | 0.964 | 0.906 | 0.013 | 0.909 | ||
MRSSL | 0.947 | 0.010 | 0.946 | 0.928 | 0.012 | 0.926 | |
0.954 | 0.009 | 0.957 | 0.939 | 0.006 | 0.939 | ||
0.960 | 0.008 | 0.960 | 0.946 | 0.010 | 0.949 | ||
0.951 | 0.005 | 0.952 | 0.944 | 0.015 | 0.944 | ||
0.951 | 0.011 | 0.950 | 0.942 | 0.009 | 0.945 | ||
0.951 | 0.011 | 0.952 | 0.942 | 0.008 | 0.941 | ||
0.949 | 0.007 | 0.947 | 0.933 | 0.010 | 0.931 |
表3 MRSL与MRSSL分别在不同MRAugment扩增等级与等级组合下的AUC的统计情况
Tab. 3 AUC statistics of MRSL with different MRAugment augmentation levels and MRSSL with different MRAugment level combinations
方法 | MRAugment等级/等级组合 | 验证集AUC | 测试集AUC | ||||
---|---|---|---|---|---|---|---|
平均值 | 标准差 | 中位数 | 平均值 | 标准差 | 中位数 | ||
MRSL | 0.936 | 0.008 | 0.938 | 0.856 | 0.012 | 0.858 | |
0.956 | 0.010 | 0.960 | 0.916 | 0.012 | 0.914 | ||
0.965 | 0.009 | 0.965 | 0.919 | 0.013 | 0.915 | ||
0.961 | 0.013 | 0.964 | 0.906 | 0.013 | 0.909 | ||
MRSSL | 0.947 | 0.010 | 0.946 | 0.928 | 0.012 | 0.926 | |
0.954 | 0.009 | 0.957 | 0.939 | 0.006 | 0.939 | ||
0.960 | 0.008 | 0.960 | 0.946 | 0.010 | 0.949 | ||
0.951 | 0.005 | 0.952 | 0.944 | 0.015 | 0.944 | ||
0.951 | 0.011 | 0.950 | 0.942 | 0.009 | 0.945 | ||
0.951 | 0.011 | 0.952 | 0.942 | 0.008 | 0.941 | ||
0.949 | 0.007 | 0.947 | 0.933 | 0.010 | 0.931 |
图3 MRSL与MRSSL在不同MRAugment扩增等级与等级组合下的AUC变化
Fig. 3 AUC changes of MRSL with different MRAugment augmentation levels and MRSSL with different MRAugment level combinations
方法 | AUC | 准确率 | 敏感性 | 特异性 | G-mean |
---|---|---|---|---|---|
两者差值 | 0.043 | 0.025 | -0.009 | 0.160 | 0.085 |
MRSL | 0.908 [0.838,0.978] | 0.858 [0.785,0.910] | 0.916 [0.843,0.957] | 0.640 [0.445,0.798] | 0.766 |
MRSSL | 0.951 [0.913,0.989] | 0.883 [0.814,0.929] | 0.905 [0.830,0.949] | 0.800 [0.609,0.911] | 0.851 |
表4 MRSL与MRSSL各自选取的代表模型在5个指标上的结果对比
Tab. 4 Result comparison between the selected models of MRSL and MRSSL on 5 evaluation indicators
方法 | AUC | 准确率 | 敏感性 | 特异性 | G-mean |
---|---|---|---|---|---|
两者差值 | 0.043 | 0.025 | -0.009 | 0.160 | 0.085 |
MRSL | 0.908 [0.838,0.978] | 0.858 [0.785,0.910] | 0.916 [0.843,0.957] | 0.640 [0.445,0.798] | 0.766 |
MRSSL | 0.951 [0.913,0.989] | 0.883 [0.814,0.929] | 0.905 [0.830,0.949] | 0.800 [0.609,0.911] | 0.851 |
方法 | AUC | 准确率 | 敏感性 | 特异性 | G-mean |
---|---|---|---|---|---|
MRSL(数据扩增) | 0.919±0.013 | 0.876±0.009 | 0.716±0.047 | 0.810±0.026 | |
MRSL(高斯噪声) | 0.887±0.019 | 0.849±0.022 | 0.905±0.029 | 0.636±0.081 | 0.757±0.046 |
PI Model | 0.890±0.023 | 0.844±0.018 | 0.882±0.034 | 0.698±0.133 | 0.780±0.062 |
FixMatch | 0.930±0.011 | 0.875±0.021 | 0.901±0.035 | 0.778±0.078 | |
UDA | 0.916±0.024 | 0.756±0.061 | 0.831±0.030 | ||
MRSSL | 0.946±0.010 | 0.893±0.013 | 0.925±0.019 | 0.843±0.025 |
表5 不同方法的结果对比
Tab. 5 Result comparison of different methods
方法 | AUC | 准确率 | 敏感性 | 特异性 | G-mean |
---|---|---|---|---|---|
MRSL(数据扩增) | 0.919±0.013 | 0.876±0.009 | 0.716±0.047 | 0.810±0.026 | |
MRSL(高斯噪声) | 0.887±0.019 | 0.849±0.022 | 0.905±0.029 | 0.636±0.081 | 0.757±0.046 |
PI Model | 0.890±0.023 | 0.844±0.018 | 0.882±0.034 | 0.698±0.133 | 0.780±0.062 |
FixMatch | 0.930±0.011 | 0.875±0.021 | 0.901±0.035 | 0.778±0.078 | |
UDA | 0.916±0.024 | 0.756±0.061 | 0.831±0.030 | ||
MRSSL | 0.946±0.010 | 0.893±0.013 | 0.925±0.019 | 0.843±0.025 |
1 | LITJENS G, KOOI T, EHTESHAMI BEJNORDI B, et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 2017, 42:60-88. 10.1016/j.media.2017.07.005 |
2 | SHEN D G, WU G R, SUK H I. Deep learning in medical image analysis[J]. Annual Review of Biomedical Engineering, 2017, 19: 221-248. 10.1146/annurev-bioeng-071516-044442 |
3 | SONG Y, ZHANG Y D, YAN X, et al. Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI[J]. Journal of Magnetic Resonance Imaging, 2018, 48(6): 1570-1577. 10.1002/jmri.26047 |
4 | HERENT P, SCHMAUCH B, JEHANNO P, et al. Detection and characterization of MRI breast lesions using deep learning[J]. Diagnostic and Interventional Imaging, 2019, 100(4): 219-225. 10.1016/j.diii.2019.02.008 |
5 | ZHOU Z Y, ZHAO G Y, KIJOWSKI R, et al. Deep convolutional neural network for segmentation of knee joint anatomy[J]. Magnetic Resonance in Medicine, 2018, 80(6): 2759-2770. 10.1002/mrm.27229 |
6 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. 10.1109/cvpr.2016.90 |
7 | HE K M, ZHANG X Y, REN S Q, et al. Identity mappings in deep residual networks[C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS9908. Cham: Springer, 2016: 630-645. |
8 | ZAGORUYKO S, KOMODAKIS N. Wide residual networks[C]// Proceedings of the 2016 British Machine Vision Conference. Durham: BMVA Press, 2016: No.87. 10.5244/c.30.87 |
9 | PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359. 10.1109/tkde.2009.191 |
10 | BIEN N, RAJPURKAR P, BALL RL, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet[J]. PLoS Medicine, 2018, 15(11): No.e1002699. 10.1371/journal.pmed.1002699 |
11 | DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 248-255. 10.1109/cvpr.2009.5206848 |
12 | ZHU X J, GOLDBERG A B. Introduction to Semi-Supervised Learning[M]. San Rafael, CA: Morgan & Claypool Publishers, 2009: 23-33. 10.3115/1621829.1621832 |
13 | LAINE S, AILA T. Temporal ensembling for semi-supervised learning[EB/OL]. (2017-03-15) [2020-12-12].. |
14 | MIYATO T, MAEDA S I, KOYAMA M, et al. Virtual adversarial training: a regularization method for supervised and semi-supervised learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8): 1979-1993. 10.1109/tpami.2018.2858821 |
15 | TARVAINEN A, VALPOLA H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates, Inc., 2017: 1195-1204. |
16 | XIE Q Z, DAI Z H, HOVY E, et al. Unsupervised data augmentation for consistency training[C/OL]// Proceedings of the 34th International Conference on Neural Information Processing Systems. [2020-12-12].. |
17 | CHEPLYGINA V, DE BRUIJNE M, PLUIM J P W. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis[J]. Medical Image Analysis, 2019, 54: 280-296. 10.1016/j.media.2019.03.009 |
18 | LIU Q D, YU L Q, LUO L Y, et al. Semi-supervised medical image classification with relation-driven self-ensembling model[J]. IEEE Transactions on Medical Imaging, 2020, 39(11): 3429-3440. 10.1109/tmi.2020.2995518 |
19 | NGUYEN H H, SAARAKKALA S, BLASCHKO M B, et al. Semixup: in- and out-of-manifold regularization for deep semi-supervised knee osteoarthritis severity grading from plain radiographs[J]. IEEE Transactions on Medical Imaging, 2020, 39: 4346-4356. 10.1109/tmi.2020.3017007 |
20 | PETERFY C G, SCHNEIDER E, NEVITT M. The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee[J]. Osteoarthritis and Cartilage, 2008, 16(12): 1433-1441. 10.1016/j.joca.2008.06.016 |
21 | HU J L, KUANG Y Z, LIAO B, et al. A multichannel 2D convolutional neural network model for task-evoked fMRI data classification[J]. Computational Intelligence and Neuroscience, 2019, 2019: No.5065214. 10.1155/2019/5065214 |
22 | MITCHELL T M. The need for biases in learning generalizations: Rutgers CS tech report CBM-TR-117[R]. New Brunswick, NJ: Rutgers University, 1980: 1-3. |
23 | CUBUK E D, ZOPH B, SHLENS J, et al. RandAugment: practical automated data augmentation with a reduced search space[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2020: 3008-3017. 10.1109/cvprw50498.2020.00359 |
24 | NYÚL L G, UDUPA J K. On standardizing the MR image intensity scale[J]. Magnetic Resonance in Medicine, 1999, 42(6): 1072-1081. 10.1002/(sici)1522-2594(199912)42:6<1072::aid-mrm11>3.0.co;2-m |
25 | LOWEKAMP B C, CHEN D T, IBÁÑEZ L, et al. The design of simpleITK[J]. Frontiers in Neuroinformatics, 2013, 7: No.45. 10.3389/fninf.2013.00045 |
26 | YANIV Z, LOWEKAMP B C, JOHNSON H J, et al. SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research[J]. Journal of Digital Imaging, 2018, 31(3): 290-303. 10.1007/s10278-017-0037-8 |
27 | WALT S VAN DER, SCHÖNBERGER J L, NUNEZ-IGLESIAS J, et al. Scikit-image: image processing in Python[J]. PeerJ, 2014, 2: No.e453. 10.7717/peerj.453 |
28 | JAPKOWICZ N, STEPHEN S. The class imbalance problem: a systematic study[J]. Intelligent Data Analysis, 2002, 6(5): 429-449. 10.3233/ida-2002-6504 |
29 | SOHN K, BERTHELOT D, LI C L, et al. FixMatch: simplifying semi-supervised learning with consistency and confidence[C/OL]// Proceedings of the 34th International Conference on Neural Information Processing Systems. [2020-12-12].. |
30 | PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[C/OL]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. [2020-12-12].. |
[1] | 马敬奇, 雷欢, 陈敏翼. 基于AlphaPose优化模型的老人跌倒行为检测算法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 294-301. |
[2] | 贾洪飞, 刘茜, 王瑜, 肖洪兵, 邢素霞. 3DPCANet在阿尔茨海默症功能磁共振成像图像分类中的应用[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 310-315. |
[3] | 谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495. |
[4] | 代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551. |
[5] | 张师鹏, 李永忠, 杜祥通. 基于半监督学习和三支决策的入侵检测模型[J]. 计算机应用, 2021, 41(9): 2602-2608. |
[6] | 毛铭泽, 曹芮浩, 闫春钢. 基于权值多样性的半监督分类算法[J]. 计算机应用, 2021, 41(9): 2473-2480. |
[7] | 郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784. |
[8] | 张笑非, 杨阳, 黄佳进, 钟宁. 基于度中心性的认知特征选择方法[J]. 计算机应用, 2021, 41(9): 2767-2772. |
[9] | 赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503. |
[10] | 徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725. |
[11] | 陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693. |
[12] | 祝承, 赵晓琦, 赵丽萍, 焦玉宏, 朱亚飞, 陈建英, 周伟, 谭颖. 基于谱聚类半监督特征选择的功能磁共振成像数据分类[J]. 计算机应用, 2021, 41(8): 2288-2293. |
[13] | 何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431. |
[14] | 曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 《计算机应用》唯一官方网站, 2021, 41(8): 2273-2287. |
[15] | 秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||