Retrieval method of pulmonary nodule images based on multi-scale convolution feature fusion

doi:10.11772/j.issn.1001-9081.2019091641

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (2): 561-565.DOI: 10.11772/j.issn.1001-9081.2019091641

• CCF Bigdata 2019 • Previous Articles Next Articles

Retrieval method of pulmonary nodule images based on multi-scale convolution feature fusion

Junhua GU¹^,²^,³, Feng WANG²^,³, Yongjun QI¹^,⁴(), Zheran SUN²^,³, Zepei TIAN²^,³, Yajuan ZHANG²^,³

^1.State Key Laboratory of Reliability and Intelligence of Electrical Equipment （Hebei University of Technology），Tianjin 300401，China
^2.Hebei Province Key Laboratory of Big Data Calculation （Hebei University of Technology），Tianjin 300401，China
^3.School of Artificial Intelligence，Hebei University of Technology，Tianjin 300401，China
^4.Information Technology Center，North China Institute of Aerospace Engineering，Langfang Hebei 065000，China

Received:2019-08-20 Revised:2019-09-26 Accepted:2019-09-26 Online:2019-10-14 Published:2020-02-10
Contact: Yongjun QI
About author:GU Junhua， born in 1966， Ph. D.， professor. His research interests include data mining， intelligent information processing.
WANG Feng， born in 1995， M. S. candidate. His research interests include medical image processing， image retrieval.
SUN Zheran， born in 1996， M. S. candidate. Her research interests include medical image processing， image classification.
TIAN Zepei， born in 1994， M. S. candidate. Her research interests include medical image processing， object detection.
ZHANG Yajuan， born in 1984， M. S. Her research interests include data mining， image processing.
Supported by:
the National Natural Science Foundation of China(61702157);the Key Program of Natural Science Foundation of Hebei Province(F2016202144)

基于多尺度卷积特征融合的肺结节图像检索方法

顾军华¹^,²^,³, 王锋²^,³, 戚永军¹^,⁴(), 孙哲然²^,³, 田泽培²^,³, 张亚娟²^,³

^1.电工设备可靠性与智能化国家重点实验室（河北工业大学），天津 300401
^2.河北省大数据计算重点实验室（河北工业大学），天津 300401
^3.河北工业大学人工智能与数据科学学院，天津 300401
^4.北华航天工业学院信息技术中心，河北廊坊 065000

通讯作者: 戚永军
作者简介:顾军华（1966—），男，天津人，教授，博士，CCF会员，主要研究方向：数据挖掘、智能信息处理
王锋（1995—），男，河北保定人，硕士研究生，主要研究方向：医学图像处理、图像检索
孙哲然（1996—），女，河北张家口人，硕士研究生，主要研究方向：医学图像处理、图像分类
田泽培（1994—），女，河北石家庄人，硕士研究生，主要研究方向：医学图像处理、目标检测
张亚娟（1984—），女，河北廊坊人，硕士，主要研究方向：数据挖掘、图像处理。
基金资助:
国家自然科学基金资助项目(61702157);河北省自然科学基金重点项目(F2016202144)

Abstract

Abstract:

In order to solve the difficulty of feature extraction and low accuracy of retrieval in pulmonary nodule image retrieval， a deep network model named LMSCRnet was proposed to extract image features. Firstly， the feature fusion method of convolution of filters with different scales was adopted to solve the problem of difficulty in obtaining local features caused by different sizes of pulmonary nodules. Then， the SE-ReSNeXt block was introduced to obtain the semantic features with higher level and reduce network degradation. Finally， the high-level semantic feature representation of pulmonary nodule image was obtained. In order to meet the needs of massive data retrieval tasks in real life， the distance calculation and sorting process were deployed on the Spark distributed platform. The experimental results show that the feature extraction method based on LMSCRnet can better extract the image high-level semantic information， and can achieve 84.48% accuracy on the preprocessed dataset of lung nodules named LIDC， and has the retrieval precision higher than other retrieval methods. At the same time， using Spark distributed platform to complete similarity matching and sorting process enables the retrieval method to meet the requirements of massive data retrieval tasks.

Key words: pulmonary nodule image, image retrieval, feature fusion, parallel optimization, Spark, deep learning

摘要：

为了解决肺结节图像检索中特征提取难度大、检索精度低下的问题，提出了一种深度网络模型——LMSCRnet用于提取图像特征。首先采用多种不同尺寸滤波器卷积的特征融合方法以解决肺结节大小不一引起的局部特征难以获取的问题，然后引入SE-ResNeXt块来得到更高级的语义特征同时减少网络退化，最后得到肺结节图像的高级语义特征表示。为满足现实中大数据量检索任务的需求，将距离计算及排序过程部署到Spark分布式平台上。实验结果表明，基于LMSCRnet的特征提取方法能够更好地提取图像高级语义信息，在肺结节预处理数据集LIDC上能够达到84.48%的准确率，检索精度高于其他检索方法，而且使用Spark分布式平台完成相似度匹配及排序过程使得检索方法能够满足大数据量检索任务需求。

关键词: 肺结节图像, 图像检索, 特征融合, 并行优化, Spark, 深度学习

CLC Number:

TP181

Junhua GU, Feng WANG, Yongjun QI, Zheran SUN, Zepei TIAN, Yajuan ZHANG. Retrieval method of pulmonary nodule images based on multi-scale convolution feature fusion[J]. Journal of Computer Applications, 2020, 40(2): 561-565.

顾军华, 王锋, 戚永军, 孙哲然, 田泽培, 张亚娟. 基于多尺度卷积特征融合的肺结节图像检索方法[J]. 《计算机应用》唯一官方网站, 2020, 40(2): 561-565.

Figures/Tables 8

Fig. 1 Pulmonary nodules of different sizes

Fig. 2 Overall network structure

Fig. 3 Structures of SE-ResNeXt and ResNeXt

Fig. 4 Parallel implementation process of the proposed algorithm

Tab. 1 Experimental results on pulmonary nodule classification

模型名称	准确率	模型名称	准确率
SIFT	45.37	M_resnet	80.32
VGG	52.01	SE-ResNeXt	75.91
CNNs	68.45	LMSCRnet	86.78

Fig. 5 Influence of feature selection on retrieval result

Tab. 2 Experimental results of pulmonary nodule retrieval

模型名称	平均查准率	模型名称	平均查准率
SIFT	39.87	M_resnet	78.32
VGG	43.56	SE-ResNeXt	72.95
CNNs	63.17	LMSCRnet	84.48

Fig. 6 Speedup of the proposed algorithm with different number of nodes

References 24

1	AKAKIN H C， GURCAN M N. Content-based microscopic image retrieval system for multi-image queries［J］. IEEE Transactions on Information Technology in Biomedicine， 2012， 16（4）： 758-769. 10.1109/titb.2012.2185829
2	COMANICIU D， MEER P， FORAN D J. Image-guided decision support system for pathology［J］. Machine Vision and Applications， 1999， 11（4）： 213-224. 10.1007/s001380050104
3	SCHNORRENBERG F， PATTICHIS C S， SCHIZAS C N， et al. Content-based retrieval of breast cancer biopsy slides［J］. Technology and Health Care， 2000， 8（5）： 291-297. 10.3233/thc-2000-8505
4	SONG L， LIU X， MA L， et al. Using HOG-LBP features and MMP learning to recognize imaging signs of lung lesions［C］// Proceedings of the 25th IEEE International Symposium on Computer-Based Medical Systems. Piscataway： IEEE， 2012： 1-4. 10.1109/cbms.2012.6266313
5	DALAL N， TRIGGS B. Histograms of oriented gradients for human detection［C］// Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2005：886-893. 10.1109/cvpr.2005.177
6	ZHI L， ZHANG S， ZHAO D， et al. Medical image retrieval using SIFT feature［C］// Proceedings of the 2nd International Congress on Image and Signal Processing. Piscataway： IEEE， 2009： 1-4. 10.1109/cisp.2009.5304112
7	LOWE D G. Distinctive image features from scale-invariant keypoints［J］. International Journal of Computer Vision， 2004， 60（2）： 91-110. 10.1023/b:visi.0000029664.99615.94
8	IBANEZ D P， LI J， SHEN Y， et al. Deep learning for pulmonary nodule CT image retrieval — an online assistance system for novice radiologists［C］// Proceedings of the 2017 IEEE International Conference on Data Mining Workshops. Piscataway： IEEE， 2017： 1112-1121. 10.1109/icdmw.2017.158
9	NISHIO M， SUGIYAMA O， YAKAMI M， et al. Computer-aided diagnosis of lung nodule classification between benign nodule， primary lung cancer， and metastatic lung cancer at different image size using deep convolutional neural network with transfer learning［J］. PloS ONE， 2018， 13（7）： No.e0200721. 10.1371/journal.pone.0200721
10	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［EB/OL］. ［2019-03-10］. . 10.5244/c.28.6
11	ZHANG G， ZHU D， LIU X， et al. Multi-scale pulmonary nodule classification with deep feature fusion via residual network［J］. Journal of Ambient Intelligence and Humanized Computing， 2018：1-12. 10.1007/s12652-018-1132-5
12	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
13	XIE S， GIRSHICK R， DOLLÁR P， et al. Aggregated residual transformations for deep neural networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5987-5995. 10.1109/cvpr.2017.634
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	SZEGEDY C， IOFFE S， VANHOUCKE V， et al. Inception-v4， inception-ResNet and the impact of residual connections on learning［C/OL］// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2017. ［2018-12-27］. .
16	Apache. Apache Spark［EB/OL］. ［2019-01-10］.. 10.1007/978-1-4842-4810-2_14
17	Apache. Apache Hadoop［EB/OL］. ［2018-12-20］.. 10.1007/978-1-4302-4864-4_20
18	Apache. Hadoop1.2.1 documentation HDFS architecture guide［EB/OL］. ［2019-02-10］.. 10.5753/wtf.2019.7717
19	KRIZHEVSKY A. Learning multiple layers of features from tiny images［R］. Toronto： University of Toronto， 2009.
20	CHEN X， FANG H， LIN T Y， et al. Microsoft COCO captions： data collection and evaluation server［EB/OL］. ［2019-01-10］. .
21	DENG J， DONG W， SOCHER R， et al. ImageNet： a large-scale hierarchical image database［C］// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2009： 248-255. 10.1109/cvpr.2009.5206848
22	ARMATO III S G， MCLENNAN G， BIDAUT L， et al. The Lung Image Database Consortium （LIDC） and Image Database Resource Initiative （IDRI）： a completed reference database of lung nodules on CT scans［J］. Medical Physics， 2011， 38（2）： 915-931. 10.1118/1.3528204
23	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et al. Dropout： a simple way to prevent neural networks from overfitting［J］. The Journal of Machine Learning Research， 2014， 15： 1929-1958.
24	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift［C］// Proceedings of the International Conference on International Conference on Machine Learning. New York： JMLR.org， 2015： 448-456.

[1]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[2]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[3]	XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation [J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.
[4]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[5]	CHEN Chengrui, SUN Ning, HE Shibiao, LIAO Yong. Deep learning-based joint channel estimation and equalization algorithm for C-V2X communications [J]. Journal of Computer Applications, 2021, 41(9): 2687-2693.
[6]	ZHENG Zhiqiang, HU Xin, WENG Zhi, WANG Yuhe, CHENG Xi. Cattle eye image feature extraction method based on improved DenseNet [J]. Journal of Computer Applications, 2021, 41(9): 2780-2784.
[7]	HE Zhenghai, XIAN Yantuan, WANG Meng, YU Zhengtao. Case reading comprehension method combining syntactic guidance and character attention mechanism [J]. Journal of Computer Applications, 2021, 41(8): 2427-2431.
[8]	ZHOU Xianbing, FAN Xiaochao, REN Ge, YANG Yong. Automated English essay scoring method based on multi-level semantic features [J]. Journal of Computer Applications, 2021, 41(8): 2205-2211.
[9]	WANG Wei, ZHAO Erping, CUI Zhiyuan, SUN Hao. Disambiguation method of multi-feature fusion based on HowNet sememe and Word2vec word embedding representation [J]. Journal of Computer Applications, 2021, 41(8): 2193-2198.
[10]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[11]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[12]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[13]	LI Yafang, LIANG Ye, FENG Weiwei, ZU Baokai, KANG Yujian. Deep network embedding method based on community optimization [J]. Journal of Computer Applications, 2021, 41(7): 1956-1963.
[14]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[15]	WU Lidan, XUE Yuyang, TONG Tong, DU Min, GAO Qinquan. Image colorization algorithm based on foreground semantic information [J]. Journal of Computer Applications, 2021, 41(7): 2048-2053.

Retrieval method of pulmonary nodule images based on multi-scale convolution feature fusion

基于多尺度卷积特征融合的肺结节图像检索方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 24

Related Articles 15

Recommended Articles

Metrics