Pulmonary nodule segmentation method based on deep transfer learning

doi:10.11772/j.issn.1001-9081.2019112012

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (7): 2117-2125.DOI: 10.11772/j.issn.1001-9081.2019112012

• Virtual reality and multimedia computing • Previous Articles Next Articles

Pulmonary nodule segmentation method based on deep transfer learning

MA Jinlin, WEI Meng, MA Ziping

School of Computer Science and Engineering, North Minzu University, Yinchuan Ningxia 750021, China

Received:2019-11-28 Revised:2020-04-01 Online:2020-07-10 Published:2020-06-29
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61462002), the National Natural Science Foundation of Ningxia Province (2020AAC3215), the "Image and Intelligent Information Processing" Innovation Team Project of State Ethnic Affairs Commission, the Innovation Team Project of "Computer Vision and Virtual Reality" of North Minzu University, the Major Project of North Minzu University (2018XYZJK02).

基于深度迁移学习的肺结节分割方法

马金林, 魏萌, 马自萍

北方民族大学计算机科学与工程学院, 银川 750021

通讯作者: 马金林
作者简介:马金林(1976-),男,宁夏银川人,副教授,博士,主要研究方向:图像处理、智能信息处理;魏萌(1996-),女,陕西咸阳人,硕士研究生,主要研究方向:图像处理;马自萍(1977-),女,宁夏银川人,副教授,博士,主要研究方向:智能信息处理。
基金资助:
国家自然科学基金资助项目（61462002）；宁夏自然科学基金资助项目（2020AAC3215）；国家民委“图像与智能信息处理”创新团队项目；北方民族大学“计算机视觉与虚拟现实”创新团队项目；北方民族大学重大专项（ZDZX201801）。

Abstract

Abstract: Focused on the issue that U-Net has a poor segmentation effect for small-volume pulmonary nodules, a segmentation method based on deep transfer learning was proposed, and Block Superimposed Fine-Tuning (BSFT) strategy was used to assist the segmentation of pulmonary nodules. Firstly, convolutional neural network was used to learn the feature information of large natural image datasets. Then, the learned features were transferred to the network for the segmentation of small pulmonary nodule image datasets. From the last sampling layer of the network, the network was released block by block and fine-tuned until the network completed the superimposition of the last layer. Finally, the similarity coefficient of Dice was quantitatively analyzed to determine the optimal segmentation network. The experimental results show that the Dice value of BSFT on LUNA16 pulmonary nodule open dataset reaches 0.917 9, which is obviously better than that of the mainstream pulmonary nodule segmentation algorithms.

Key words: pulmonary nodule segmentation, deep transfer learning, Convolutional Neural Network (CNN), U-Net segmentation network

摘要： 针对U-Net分割小体积肺结节效果较差的问题，提出一种基于深度迁移学习的分割方法，利用分块式叠加微调（BSFT）策略辅助分割肺结节。首先，利用卷积神经网络学习自然图像大数据集的特征信息；然后，将所学特征迁移到进行肺结节图像小数据集分割的网络，从该网络最后一个下采样层开始逐块释放、微调训练，直到网络完成最后一层的叠加；最后，定量分析Dice相似性系数，以确定最佳分割网络。实验结果表明，BSFT在LUNA16肺结节公开数据集上的Dice值达到0.917 9，该策略的性能明显优于主流肺结节分割算法。

关键词: 肺结节分割, 深度迁移学习, 卷积神经网络, U-Net分割网络

CLC Number:

TP391

MA Jinlin, WEI Meng, MA Ziping. Pulmonary nodule segmentation method based on deep transfer learning[J]. Journal of Computer Applications, 2020, 40(7): 2117-2125.

马金林, 魏萌, 马自萍. 基于深度迁移学习的肺结节分割方法[J]. 计算机应用, 2020, 40(7): 2117-2125.

References

[1] ZHU Z,DONG S,YU C,et al. A text hybrid clustering algorithm based on HowNet semantics[J]. Key Engineering Materials,2011, 474/475/476/476:2071-2078.
[2] 张花齐, 王光磊, 李艳, 等. CT图像肺结节的全自动算法研究[J]. 激光杂志,2019,40(4):59-63.(ZHANG H Q,WANG G L,LI Y,et al. Automated algorithm for CT image of pulmonary nodules[J]. Laser Journal,2019,40(4):59-63.)
[3] 孙申申, 李宏, 侯欣然, 等. 基于EM和Mean-shift的肺结节分割[J]. 中国图象图形学报,2009,14(10):2016-2022.(SUN S S, LI H,HOU X R,et al. Pulmonary nodule segmentation based on EM and Mean-shift[J]. Journal of Image and Graphics,2009,14(10):2016-2022.)
[4] ARMATO S G II,GIGER M L,MORAN C J,et al. Computerized detection of pulmonary nodules on CT scans[J]. RadioGraphics, 1999,19(5):1301-1311.
[5] KANAZAWA K,KAWATA Y,NIKI N,et al. Computer-aided diagnosis for pulmonary nodules based on helical CT images[J]. Computerized Medical Imaging and Graphics,1998,22(2):157-167.
[6] MIWA T,KAKO J-I,YAMAMOTO S,et al. Automatic detection of lung cancers in chest CT images by the variable N-quoit filter[J]. Systems and Computers in Japan,2002,33(1):53-63.
[7] MESSAY T,HARDIE R C,TUINSTRA T R. Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the lung image database consortium and image database resource initiative dataset[J]. Medical Image Analysis,2015,22(1):48-62.
[8] ZHAO J,JI G,XIA Y,et al. Cavitary nodule segmentation in computed tomography images based on self-generating neural networks and particle swarm optimisation[J]. International Journal of Bio-Inspired Computation,2015,7(1):62-67.
[9] MONKAM P,QI S,XU M,et al. CNN models discriminating between pulmonary micro-nodules and non-nodules from CT images[J]. BioMedical Engineering OnLine,2018,17(1):No. 96.
[10] SHELHAMER E,LONG J,DARRELT. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651.
[11] CUI S,MAO L,XIONG S. Brain tumor automatic segmentation using fully convolutional networks[J]. Journal of Medical Imaging and Health Informatics,2017,7(7):1641-1647.
[12] SHAIKH M,ANAND G,ACHARYA G,et al. Brain tumor segmentation using dense fully convolutional neural network[C]//Proceedings of the 3rd International MICCAI Brainlesion Workshop. Cham:Springer,2017:309-319.
[13] RONNEBERGER O,FISCHER P,BROX T. U-Net:convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention,LNCS 9351. Cham:Springer,2015:234-241.
[14] TONG G,LI Y,CHEN H,et al. Improved U-NET network for pulmonary nodules segmentation[J]. Optik, 2018, 174:460-469.
[15] LIU H,CAO H,SONG E M. A cascaded dual-pathway residual network for lung nodule segmentation in CT images[J]. Physica Medica,2019,63:112-121.
[16] 张声超. 基于特征融合的肺结节检测关键技术研究[D]. 广州:华南理工大学,2018:33-43. (ZHANG S C. Research on key techniques of pulmonary nodule detection based on feature fusion[D]. Guangzhou:South China University of Technology,2018:33-43.)
[17] PAN S J,YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[18] OQUAB M,BOTTOU L,LAPTEV I,et al. Learning and transferring mid-level image representations using convolutional neural networks[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:1717-1724.
[19] 杨涵方, 周向东. 基于深度稀疏辨别的跨领域图像分类[J]. 计算机工程,2018,44(4):310-316. (YANG H F,ZHOU X D. Cross domain image classification based on deep sparse discrimination[J]. Computer Engineering,2018,44(4):310-316.)
[20] 李浩波, 祝忠明, 范治中. 基于深度学习的腺体分化程度自动分类[J]. 信息技术与信息化,2019(6):93-94. (LI H B,ZHU Z M,FAN Z Z. Automatic classification of glandular differentiation based on deep learning[J]. Information Technology and Informatization,2019(6):93-94.
[21] 李淼, 王敬贤, 李华龙, 等. 基于CNN和迁移学习的农作物病害识别方法研究[J]. 智慧农业,2019,1(3):46-55.(LI M, WANG J X,LI H L,et al. Method for identifying crop disease based on CNN and transfer learning[J]. Smart Agriculture,2019,1(3):46-55.)
[22] 徐胜舟, 程时宇. 基于全卷积神经网络迁移学习的乳腺肿块图像分割[J]. 中南民族大学学报(自然科学版),2019,38(2):278-284. (XU S Z,CHENG S Y. Breast mass image segmentation based on transfer learning of fully convolutional neural networks[J]. Journal of South-Central University for Nationalities (Natural Science Edition),2019,38(2):278-284.)
[23] TAJBAKHSH N,SHIN J Y,GURUDU S,et al. Convolutional neural networks for medical image analysis:full training or fine tuning?[J]. IEEE Transactions on Medical Imaging,2016,35(5):1299-1312.
[24] 褚晶辉, 吴泽蕤, 吕卫, 等. 基于迁移学习和深度卷积神经网络的乳腺肿瘤诊断系统[J]. 激光与光电子学进展,2018,55(8):No. 081001. (CHU J H,WU Z R,LYU W, et al. Breast cancer diagnosis system based on transfer learning and deep convolutional neural networks[J]. Laser & Optoelectronics Progress,2018,55(8):No. 081001.)
[25] SHIN H C,ROTH H R, GAO M,et al. Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J]. IEEE Transactions on Medical Imaging,2016,35(5):1285-1298.
[26] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2018-12-01]. https://arxiv.org/pdf/1409.1556.pdf.
[27] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:770-778.
[28] SZEGEDY C,VANHOUCKE V,IOFFE S,et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the 2016 IEEE Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:2818-2826.
[29] HUANG G,LIU Z,VAN DER MAATEN L,et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2261-2269.
[30] KINGMA D P,BA J L. Adam:a method for stochastic optimization[EB/OL].[2018-12-01]. https://arxiv.org/pdf/1412.6980.pdf.
[31] WANG S,ZHOU M,LIU Z,et al. Central focused convolutional neural network:developing a data-driven mode for lung nodule segmentation[J]. Medical Image Analysis,2017,40:172-183.
[32] MILLETARI F,NAVAB N,AHMADI S A. V-Net:fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of the 4th International Conference on 3D Vision. Piscataway:IEEE,2016:565-571.

Pulmonary nodule segmentation method based on deep transfer learning

基于深度迁移学习的肺结节分割方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[2]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[8]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[9]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[10]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[11]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[12]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[13]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.
[14]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[15]	ZHAO Xiaohu, LI Xiao. Image captioning algorithm based on multi-feature extraction [J]. Journal of Computer Applications, 2021, 41(6): 1640-1646.