Interstitial lung disease segmentation algorithm based on multi-task learning

doi:10.11772/j.issn.1001-9081.2023040517

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1285-1293.DOI: 10.11772/j.issn.1001-9081.2023040517

• Multimedia computing and computer simulation • Previous Articles Next Articles

Interstitial lung disease segmentation algorithm based on multi-task learning

Wei LI¹, Ling CHEN², Xiuyuan XU¹, Min ZHU², Jixiang GUO¹, Kai ZHOU¹, Hao NIU¹, Yuchen ZHANG³, Shanye YI³, Yi ZHANG¹, Fengming LUO²()

^1.College of Computer Science，Sichuan University，Chengdu Sichuan 610065，China
^2.West China Hospital，Sichuan University，Chengdu Sichuan 610044，China
^3.West China School of Medicine，Sichuan University，Chengdu Sichuan 610041，China

Received:2023-05-04 Revised:2023-07-13 Accepted:2023-07-14 Online:2023-08-03 Published:2024-04-10
Contact: Fengming LUO
About author:LI Wei， born in 1994， M. S. candidate. His research interests include medical image segmentation.
CHEN Ling， born in 1989， Ph. D.， assistant research fellow. Her research interests include interstitial lung disease.
XU Xiuyuan， born in 1992， Ph. D.， associate research fellow. His research interests include neural network， intelligent medicine.
ZHU Min， born in 1989， Ph. D.， assistant research fellow. Her research interests include interstitial lung disease.
GUO Jixiang， born in 1981， Ph. D.， associate professor. Her research interests include neural network， intelligent medicine.
ZHOU Kai， born in 1995， Ph. D. candidate. His research interests include neural network， intelligent medicine.
NIU Hao， born in 1983， Ph. D. candidate. His research interests include neural network， intelligent medicine.
ZHANG Yuchen， born in 1996， M. S. candidate. Her research interests include interstitial lung disease.
YI Shanye， born in 1998， M. S. candidate. Her research interests include general practice.
ZHANG Yi， born in 1963， Ph. D.， professor. His research interests include artificial intelligence， neural network.
LUO Fengming， born in 1970， Ph. D.， professor. His research interests include interstitial lung disease， plateau respiratory chronic disease， respiratory intervention technology.
Supported by:
National Natural Science Foundation of China(62106163);CAAI-Huawei MindSpore Open Fund(21H1235)

基于多任务学习的间质性肺病分割算法

李威¹, 陈玲², 徐修远¹, 朱敏², 郭际香¹, 周凯¹, 牛颢¹, 张煜宸³, 易珊烨³, 章毅¹, 罗凤鸣²()

^1.四川大学计算机学院，成都 610065
^2.四川大学华西医院，成都 610044
^3.四川大学华西临床医学院，成都 610041

通讯作者: 罗凤鸣
作者简介:李威（1994—），男，四川绵竹人，硕士研究生，CCF会员，主要研究方向：医学图像分割
陈玲（1989—），女，江西抚州人，助理研究员，博士，主要研究方向：间质性肺病
徐修远（1992—），男，安徽合肥人，副研究员，博士，主要研究方向：神经网络、智能医学
朱敏（1989—），女，四川成都人，助理研究员，博士，主要研究方向：间质性肺病
郭际香（1981—），女，山西吕梁人，副教授，博士，CCF会员，主要研究方向：神经网络、智能医学
周凯（1995—），男，四川眉山人，博士研究生，主要研究方向：神经网络、智能医学
牛颢（1983—），男，四川成都人，博士研究生，主要研究方向：神经网络、智能医学
张煜宸（1996—），女，浙江台州人，硕士研究生，主要研究方向：间质性肺病
易珊烨（1998—），女，四川乐山人，硕士研究生，主要研究方向：全科医学
章毅（1963—），男，四川成都人，教授，博士，主要研究方向：人工智能、神经网络
罗凤鸣（1970—），男，四川成都人，教授，博士，主要研究方向：间质性肺病、高原呼吸慢病、呼吸介入技术。fengmingluo@outlook.com
基金资助:
国家自然科学基金资助项目(62106163);中国人工智能学会-华为MindSpore学术奖励基金资助项目(21H1235)

Abstract

Abstract:

Interstitial Lung Disease （ILD） segmentation labels are highly costly， leading to small sample sizes in existing datasets and resulting in poor performance of trained models. To address this issue， a segmentation algorithm for ILD based on multi-task learning was proposed. Firstly， a multi-task segmentation model was constructed based on U-Net. Then， the generated lung segmentation labels were used as auxiliary task labels for multi-task learning. Finally， a method of dynamically weighting the multi-task loss functions was used to balance the losses of the primary task and the secondary task. Experimental results on a self-built ILD dataset show that the Dice Similarity Coefficient （DSC） of the multi-task segmentation model reaches 82.61%， which is 2.26 percentage points higher than that of U-Net. The experimental results demonstrate that the proposed algorithm can improve the segmentation performance of ILD and can assist clinical doctors in ILD diagnosis.

Key words: Interstitial Lung Disease (ILD), semantic segmentation, small sample size, multi-task learning, adaptive multi-task loss function

摘要：

间质性肺病（ILD）的分割标签标注成本极高，且现有数据集通常存在样本量较少的问题，导致训练的模型效果较差。针对该问题，提出一种基于多任务学习的ILD分割算法。首先，基于U-Net构建多任务分割模型；其次，使用生成的肺部分割标签作为辅助任务标签进行多任务学习；最后，使用一种自适应调整多任务损失函数权重的方法，平衡主任务和辅助任务的损失。在自构建的ILD数据集上的实验结果表明，多任务分割模型的Dice相似系数（DSC）达到了82.61%，与U-Net相比提升了2.26个百分点。验证了所提算法可以提升ILD的分割性能，协助临床医生进行ILD诊断。

关键词: 间质性肺病, 语义分割, 小样本量, 多任务学习, 自适应多任务损失函数

CLC Number:

TP391.4

Wei LI, Ling CHEN, Xiuyuan XU, Min ZHU, Jixiang GUO, Kai ZHOU, Hao NIU, Yuchen ZHANG, Shanye YI, Yi ZHANG, Fengming LUO. Interstitial lung disease segmentation algorithm based on multi-task learning[J]. Journal of Computer Applications, 2024, 44(4): 1285-1293.

李威, 陈玲, 徐修远, 朱敏, 郭际香, 周凯, 牛颢, 张煜宸, 易珊烨, 章毅, 罗凤鸣. 基于多任务学习的间质性肺病分割算法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1285-1293.

Figures/Tables 12

References 29

1	CHOI Y， LIU T T， PANKRATZ D G， et al. Identification of usual interstitial pneumonia pattern using RNA-Seq and machine learning： challenges and solutions［J］. BMC Genomics， 2018， 19（）： 101. 10.1186/s12864-018-4467-6
2	British Thoracic Society， Standards of Care Committee. The diagnosis， assessment and treatment of diffuse parenchymal lung disease in adults［J］. Thorax， 1999， 54（）： S1-S14. 10.1136/thx.54.suppl_1.s1
3	何渝娇，李风森.间质性肺病治疗的研究进展［J］.新疆中医药，2019，37（6）：102-105.
	HE Y J， LI F S. Research progress in the treatment of interstitial lung disease ［J］. Xinjiang Journal of Traditional Chinese Medicine， 2019， 37（6）： 102-105.
4	WIDELL J， LIDÉN M. Interobserver variability in high-resolution CT of the lungs［J］. European Journal of Radiology Open， 2020， 7： 100228. 10.1016/j.ejro.2020.100228
5	施俊，汪琳琳，王珊珊，等.深度学习在医学影像中的应用综述［J］.中国图象图形学报，2020，25（10）：1953-1981. 10.11834/jig.200255
	SHI J， WANG L L， WANG S S， et al. Applications of deep learning in medical imaging： a survey ［J］. Journal of Image and Graphics， 2020， 25（10）： 1953-1981. 10.11834/jig.200255
6	曹玉红，徐海，刘荪傲，等.基于深度学习的医学影像分割研究综述［J］.计算机应用，2021，41（8）：2273-2287. 10.11772/j.issn.1001-9081.2020101638
	CAO Y H， XU H， LIU S A， et al. Review of deep learning-based medical image segmentation ［J］. Journal of Computer Applications， 2021， 41（8）： 2273-2287. 10.11772/j.issn.1001-9081.2020101638
7	AGARWALA S， KALE M， KUMAR D， et al. Deep learning for screening of interstitial lung disease patterns in high-resolution CT images［J］. Clinical Radiology， 2020， 75（6）： 481. 10.1016/j.crad.2020.01.010
8	BUONGIORNO R， GERMANESE D， ROMEI C， et al. UIP-Net： a decoder-encoder CNN for the detection and quantification of usual interstitial pneumoniae pattern in lung CT scan images［C］// Proceedings of the 2021 International Conference on Pattern Recognition. Cham： Springer， 2021： 389-405. 10.1007/978-3-030-68763-2_30
9	VAKALOPOULOU M， CHASSAGNON G， PARAGIOS N， et al. Deep patch-based priors under a fully convolutional encoder-decoder architecture for interstitial lung disease segmentation［C］// Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging. Piscataway： IEEE， 2018： 855-858. 10.1109/isbi.2018.8363706
10	DEPEURSINGE A， VARGAS A， PLATON A， et al. Building a reference multimedia database for interstitial lung diseases［J］. Computerized Medical Imaging and Graphics， 2012， 36（3）： 227-238. 10.1016/j.compmedimag.2011.07.003
11	RUDER S. An overview of multi-task learning in deep neural networks ［EB/OL］. （2017-06-15）［2023-05-01］. .
12	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. 10.1007/978-3-319-24574-4_28
13	ZHANG Y， YANG Q. A survey on multi-task learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2022， 34（12）： 5586-5609. 10.1109/tkde.2021.3070203
14	CIPOLLA R， GAL Y， KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7482-7491. 10.1109/cvpr.2018.00781
15	HARRISON A P， XU Z， GEORGE K， et al. Progressive and multi-path holistically nested neural networks for pathological lung segmentation from CT images［C］// Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention. Cham： Springer， 2017： 621-629. 10.1007/978-3-319-66179-7_71
16	KHANNA A， LONDHE N D， GUPTA S， et al. A deep residual U-Net convolutional neural network for automated lung segmentation in computed tomography images［J］. Biocybernetics and Biomedical Engineering， 2020， 40（3）： 1314-1327. 10.1016/j.bbe.2020.07.007
17	LIU C， ZHAO R， PANG M. Lung segmentation based on random forest and multi-scale edge detection［J］. IET Image Processing， 2019， 13（10）： 1745-1754. 10.1049/iet-ipr.2019.0130
18	AGARWALA S， KUMAR A， DHARA A K， et al. Special convolutional neural network for identification and positioning of interstitial lung disease patterns in computed tomography images［J］. Pattern Recognition and Image Analysis， 2021， 31： 730-738. 10.1134/s1054661821040027
19	THORAT O， SALVI S， DEDHIA S， et al. Domain adaptation and weight initialization of neural networks for diagnosing interstitial lung diseases［J］. International Journal of Imaging Systems and Technology， 2022， 32（5）： 1535-1547. 10.1002/ima.22716
20	JUNAYED M S， JENY A A， ISLAM M B， et al. An efficient end-to-end deep neural network for interstitial lung disease recognition and classification［J］. Turkish Journal of Electrical Engineering and Computer Sciences， 2022， 30（4）： 1235-1250. 10.55730/1300-0632.3846
21	ANTHIMOPOULOS M， CHRISTODOULIDIS S， EBNER L， et al. Semantic segmentation of pathological lung tissue with dilated fully convolutional networks［J］. IEEE journal of Biomedical and Health Informatics， 2019， 23（2）： 714-722. 10.1109/jbhi.2018.2818620
22	LIAO F， LIANG M， LI Z， et al. Evaluate the malignancy of pulmonary nodules using the 3-D deep leaky noisy-OR network［J］. IEEE Transactions on Neural Networks and Learning Systems， 2019， 30（11）： 3484-3495. 10.1109/tnnls.2019.2892409
23	XU X， WANG C， GUO J， et al. DeepLN： a framework for automatic lung nodule detection using multi-resolution CT screening images［J］. Knowledge-Based Systems， 2020， 189： 105128. 10.1016/j.knosys.2019.105128
24	KINGMA D P， BA J. Adam： a method for stochastic optimization ［EB/OL］. （2017-01-30）［2023-05-01］. .
25	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. 10.1109/tpami.2016.2644615
26	CHEN L-C， ZHU Y， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 833-851. 10.1007/978-3-030-01234-2_49
27	CHEN J， LU Y， YU Q， et al. TransUNet： Transformers make strong encoders for medical image segmentation［EB/OL］. （2021-02-08）［2023-05-01］. . 10.48550/arXiv.2102.04306
28	CAO H， WANG Y， CHEN J， et al. Swin-Unet： Unet-like pure transformer for medical image segmentation［C］// Proceedings of the 2022 European Conference on Computer Vision. Cham： Springer， 2023： 205-218. 10.1007/978-3-031-25066-8_9
29	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 3431-3440. 10.1109/cvpr.2015.7298965

编号	模型	设置	DSC
对照组1	U-Net	单任务	80.35
对照组2	U-Net	预训练+微调	81.18
对照组3	多任务分割模型	多数据集预训练+微调	81.90
对照组4	U-Net	肺分割过滤输入	79.06
对照组5	多任务分割模型	单任务	80.26
实验组	多任务分割模型	伪标签+多任务	82.61

编号	模型	设置	DSC
对照组1	U-Net	单任务	80.35
对照组2	U-Net	预训练+微调	81.18
对照组3	多任务分割模型	多数据集预训练+微调	81.90
对照组4	U-Net	肺分割过滤输入	79.06
对照组5	多任务分割模型	单任务	80.26
实验组	多任务分割模型	伪标签+多任务	82.61

主任务权重	辅助任务权重	DSC/%
0.95	0.05	81.42
0.90	0.10	81.95
0.80	0.20	82.15
0.70	0.30	82.12
0.60	0.40	81.56
0.50	0.50	81.11
0.40	0.60	80.77
0.20	0.80	79.53
0.05	0.95	77.80
自适应	自适应	82.61

主任务权重	辅助任务权重	DSC/%
0.95	0.05	81.42
0.90	0.10	81.95
0.80	0.20	82.15
0.70	0.30	82.12
0.60	0.40	81.56
0.50	0.50	81.11
0.40	0.60	80.77
0.20	0.80	79.53
0.05	0.95	77.80
自适应	自适应	82.61

模型	DSC
SegNet^［25］	76.42
DeepLabv3+^［26］	78.95
PNet^［7］	79.35
Swin-Unet^［28］	72.41
TransUNet^［27］	76.67
本文模型	82.61

Interstitial lung disease segmentation algorithm based on multi-task learning

基于多任务学习的间质性肺病分割算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 29

Related Articles 15

Recommended Articles

Metrics

模型	DSC
模型	20%样本量	50%样本量	100%样本量
PNet^［7］	75.78	78.69	79.35
本文模型	80.04	81.24	82.61

[1]	Pengfei ZHANG, Litao HAN, Hengjian FENG, Hongmei LI. Point cloud semantic segmentation based on attention mechanism and global feature optimization [J]. Journal of Computer Applications, 2024, 44(4): 1086-1092.
[2]	Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316.
[3]	Aiguo SHANG, Xinjuan ZHU. Joint approach of intent detection and slot filling based on multi-task learning [J]. Journal of Computer Applications, 2024, 44(3): 690-695.
[4]	Ning WU, Yangyang LUO, Huajie XU. Semantic segmentation method for remote sensing images based on multi-scale feature fusion [J]. Journal of Computer Applications, 2024, 44(3): 737-744.
[5]	Cunyi LIAO, Yi ZHENG, Weijin LIU, Huan YU, Shouyin LIU. Decoupling-fusing algorithm for multiple tasks with autonomous driving environment perception [J]. Journal of Computer Applications, 2024, 44(2): 424-431.
[6]	Yudan SONG, Jing WANG, Xuehui WANG, Zhaoyang MA, Youfang LIN. Sleep physiological time series classification method based on adaptive multi-task learning [J]. Journal of Computer Applications, 2024, 44(2): 654-662.
[7]	Yongjiang LIU, Bin CHEN. Pixel-level unsupervised industrial anomaly detection based on multi-scale memory bank [J]. Journal of Computer Applications, 2024, 44(11): 3587-3594.
[8]	Ziyi LI, Tingting QU, Qianpeng CHONG, Jindong XU. Remote sensing image segmentation network based on fuzzy multiscale features [J]. Journal of Computer Applications, 2024, 44(11): 3581-3586.
[9]	Qiumei ZHENG, Weiwei NIU, Fenghua WANG, Dan ZHAO. Dual-branch real-time semantic segmentation network based on detail enhancement [J]. Journal of Computer Applications, 2024, 44(10): 3058-3066.
[10]	Xiao GUO, Yanping CHEN, Ruixue TANG, Ruizhang HUANG, Yongbin QIN. Multi-task learning model for charge prediction with action words [J]. Journal of Computer Applications, 2024, 44(1): 159-166.
[11]	Di ZHOU, Zili ZHANG, Jia CHEN, Xinrong HU, Ruhan HE, Jun ZHANG. Stomach cancer image segmentation method based on EfficientNetV2 and object-contextual representation [J]. Journal of Computer Applications, 2023, 43(9): 2955-2962.
[12]	Shuai ZHENG, Xiaolong ZHANG, He DENG, Hongwei REN. 3D liver image segmentation method based on multi-scale feature fusion and grid attention mechanism [J]. Journal of Computer Applications, 2023, 43(7): 2303-2310.
[13]	Bin LU, Jielin LIU. Semantic segmentation for 3D point clouds based on feature enhancement [J]. Journal of Computer Applications, 2023, 43(6): 1818-1825.
[14]	Jianhui HE, Chunlong HU, Xin SHU. Multi-task age estimation method based on multi-peak label distribution learning [J]. Journal of Computer Applications, 2023, 43(5): 1578-1583.
[15]	Quan YUAN, Yunpeng XU, Chengliang TANG. Document-level relation extraction method based on path labels [J]. Journal of Computer Applications, 2023, 43(4): 1029-1035.