Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution

doi:10.11772/j.issn.1001-9081.2020060783

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (3): 891-897.DOI: 10.11772/j.issn.1001-9081.2020060783

Special Issue: 前沿与综合应用

• Frontier and comprehensive applications • Previous Articles Next Articles

Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution

HU Yishan^1,2,3, QIN Pinle^1,2,3, ZENG Jianchao^1,2,3, CHAI Rui^1,2,3, WANG Lifang^1,2,3

1. Shanxi Medical Imaging and Data Analysis Engineering Research Center(North University of China), Taiyuan Shanxi 030051, China;
2. School of Data Science and Technology, North University of China, Taiyuan Shanxi 030051, China;
3. Shanxi Medical Imaging Artificial Intelligence Engineering Technology Research Center(North University of China), Taiyuan Shanxi 030051, China

Received:2020-06-09 Revised:2020-08-09 Online:2021-03-10 Published:2020-12-22
Supported by:
This work is partially supported by the Innovation Program of Graduate Student Education of Shanxi Province (2020SY381).

基于特征融合和动态多尺度空洞卷积的超声甲状腺分割网络

胡屹杉^1,2,3, 秦品乐^1,2,3, 曾建潮^1,2,3, 柴锐^1,2,3, 王丽芳^1,2,3

1. 山西省医学影像与数据分析工程研究中心(中北大学), 太原 030051;
2. 中北大学大数据学院, 太原 030051;
3. 山西省医学影像人工智能工程技术研究中心(中北大学), 太原 030051

通讯作者: 曾建潮
作者简介:胡屹杉(1996-),男,山西运城人,硕士研究生,主要研究方向:机器学习、计算机视觉、医学影像分析;秦品乐(1978-),男,山西长治人,教授,博士,CCF会员,主要研究方向:医学影像分析、大数据、机器视觉;曾建潮(1963-),男,山西太原人,教授,博士生导师,博士,CCF会员,主要研究方向:复杂系统的维护决策、健康管理;柴锐(1985-),男,山西运城人,讲师,博士,主要研究方向:医学影像处理;王丽芳(1977-),女,山西长治人,副教授,博士,主要研究方向:机器视觉、大数据、医学图像处理。
基金资助:
山西省研究生教育创新项目（2020SY381）。

Abstract

Abstract: Concerning the the size and morphological diversity of thyroid tissue and the complexity of surrounding tissue in thyroid ultrasound images, an ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution was proposed. Firstly, the dilated convolutions with different dilation rates and dynamic filters were used to fuse the global semantic information of different receptive domains and the semantic information in the context details with different ranges, so as to improve the adaptability and accuracy of the network to multi-scale targets. Then, the hybrid upsampling method was used to enhance the spatial information of high-dimensional semantic features and the context information of low-dimensional spatial features during feature dimensionality reduction. Finally, the spatial attention mechanism was introduced to optimize the low-dimensional features of the image, and the method of fusing high- and low-dimensional features was applied to retain the useful features of high- and low-dimensional feature information with the elimination of the redundant information and improve the network's ability to distinguish the background and foreground of the image. Experimental results show that the proposed method has an accuracy rate of 0.963±0.026, a recall rate of 0.84±0.03 and a dice coefficient of 0.79±0.03 in the public dataset of thyroid ultrasound images. It can be seen that the proposed method can solve the problems of large difference of tissue morphology and complex surrounding tissues.

Key words: image segmentation, attention mechanism, dilated convolution, ultrasound image, feature fusion

摘要： 针对甲状腺超声影像中甲状腺组织大小和形态的多样性以及周边组织的复杂性，提出了一种基于特征融合和动态多尺度空洞卷积的超声甲状腺分割网络。首先，利用不同膨胀率的空洞卷积和动态滤波器来融合不同感受野下的全局语义特征与不同范围的上下文详情的语义特征，从而提升网络对多尺度目标的适应性与准确度；然后，在特征降维时采用混合上采样方式，以增强高维语义特征的空间信息和低维空间特征的上下文信息；最后，采用空间注意力机制来优化图像的低维特征，并采用高低维特征融合的方式使高低维特征信息在保留重要特征的同时摒弃冗余信息以及使网络对于图像前背景的区分能力得到增强。实验结果表明，所提方法在甲状腺超声影像公开数据集上达到了0.963±0.026的准确率、0.84±0.03的召回率和0.79±0.03的dice系数。可见所提方法能较好地解决组织形态差异性大以及周边组织复杂的问题。

关键词: 图像分割, 注意力机制, 空洞卷积, 超声影像, 特征融合

CLC Number:

TP183

HU Yishan, QIN Pinle, ZENG Jianchao, CHAI Rui, WANG Lifang. Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution[J]. Journal of Computer Applications, 2021, 41(3): 891-897.

胡屹杉, 秦品乐, 曾建潮, 柴锐, 王丽芳. 基于特征融合和动态多尺度空洞卷积的超声甲状腺分割网络[J]. 计算机应用, 2021, 41(3): 891-897.

References

[1] 曾倩, 崔芳芳, 宇传华, 等. 中国癌症发病、死亡现状与趋势分析[J]. 中国卫生统计,2016,33(2):321-323.(ZENG Q,CUI F F, YU C H,et al. Analysis of current status and trend of cancer incidence and death in China[J]. Chinese Journal of Health Statistics,2016,33(2):321-323.)
[2] QUAN L,ZHANG D,YANG Y,et al. Segmentation of tumor ultrasound image via region-based Ncut method[J]. Wuhan University Journal of Natural Sciences,2013,18(4):313-318
[3] ZHUANG Z,LEI N,RAJ A N,et al. Application of fractal theory and fuzzy enhancement in ultrasound image segmentation[J]. Medical and Biological Engineering and Computing,2019,57(3):623-632.
[4] ALOM M Z, YAKOPCIC C, TAHA T M, et al. Nuclei segmentation with Recurrent Residual convolutional neural networks based U-Net(R2U-Net)[C]//Proceedings of 2018 IEEE National Aerospace and Electronics Conference. Piscataway:IEEE,2018:228-233.
[5] NANDAMURI S,CHINA D,MITRA P,et al. SUMNet:fully convolutional model for fast segmentation of anatomical structures in ultrasound volumes[C]//Proceedings of the IEEE 16th International Symposium on Biomedical Imaging. Piscataway:IEEE,2019:1729-1732.
[6] 迟剑宁, 于晓升, 张艺菲. 融合深度网络和浅层纹理特征的甲状腺结节癌变超声图像诊断[J]. 中国图象图形学报,2018,23(10):1582-1593.(CHI J N,YU X S,ZHANG Y F. Thyroid nodule malignantrisk detection in ultrasound image by fusing deep and texture features[J]. Journal of Image and Graphics,2018,23(10):1582-1593.)
[7] ZHAO T,WU X. Pyramid feature attention network for saliency detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:3080-3089.
[8] QIN P,WU K,HU Y,et al. Diagnosis of benign and malignant thyroid nodules using combined conventional ultrasound and ultrasound elasticity imaging[J]. IEEE Journal of Biomedical and Health Informatics,2020,24(4):1028-1036.
[9] CHEN S, TAN X, WANG B, et al. Reverse attention-based residual network for salient object detection[J]. IEEE Transactions on Image Processing,2020,29:3763-3776.
[10] 李晓光, 付陈平, 李晓莉, 等. 面向多尺度目标检测的改进Faster R-CNN算法[J]. 计算机辅助设计与图形学学报,2019, 31(7):1095-1101.(LI X G,FU C P,LI X L,et al. Improved Faster R-CNN for multi-scale object detection[J]. Journal of Computer Aided-Design and Computer Graphics,2019,31(7):1095-1101.)
[11] CHEN L C,ZHU Y,PAPANDREOU G,et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:833-851.
[12] LIN T Y,DOLLÁR P,GIRSHICK R,et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:936-944.
[13] CHEN L C,PAPANDREOU G,KOKKINOS I,et al. DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[14] ZHAO H,SHI J,QI X,et al. Pyramid scene parsing network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:6230-6239.
[15] PENG C,ZHANG X,YU G,et al. Large kernel matters-improve semantic segmentation by global convolutional network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:1743-1751.
[16] WEI J,WANG S,HUNAG Q. F3Net:fusion,feedback and focus for salient object detection[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press,2020:12321-12328.
[17] WANG Z,SIMONCELLI E P,BOVIK A C. Multiscale structural similarity for image quality assessment[C]//Proceedings of the 37th Asilomar Conference on Signals,Systems and Computers. Piscataway:IEEE,2003:1398-1402.
[18] WUNDERLING T,GOLLA B,POUDEL P,et al. Comparison of thyroid segmentation techniques for 3D ultrasound[C]//Proceedings of the SPIE 10133,Medical Imaging 2017:Image Processing. Bellingham,WA:SPIE,2017:No. 1013317.
[19] GU Z,CHENG J,FU H,et al. CE-Net:context encoder network for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging,2019,38(10):2281-2292.
[20] QIN X,ZHANG Z,HUANG C,et al. BASNet:boundary-aware salient object detection[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:7471-7481.

Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution

基于特征融合和动态多尺度空洞卷积的超声甲状腺分割网络

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[2]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[3]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[4]	LIU Yaxuan, ZHONG Yong. Joint extraction method of entities and relations based on subject attention [J]. Journal of Computer Applications, 2021, 41(9): 2517-2522.
[5]	YANG Rui, QIAN Xiaojun, SUN Zhenqiang, XU Zhen. Hybrid aerial image segmentation algorithm based on multi-region feature fusion for natural scene [J]. Journal of Computer Applications, 2021, 41(8): 2445-2452.
[6]	ZHOU Xianbing, FAN Xiaochao, REN Ge, YANG Yong. Automated English essay scoring method based on multi-level semantic features [J]. Journal of Computer Applications, 2021, 41(8): 2205-2211.
[7]	WANG Wei, ZHAO Erping, CUI Zhiyuan, SUN Hao. Disambiguation method of multi-feature fusion based on HowNet sememe and Word2vec word embedding representation [J]. Journal of Computer Applications, 2021, 41(8): 2193-2198.
[8]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[9]	DANG Weichao, LI Tao, BAI Shangwang, GAO Gaimei, LIU Chunxia. Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network [J]. Journal of Computer Applications, 2021, 41(8): 2346-2351.
[10]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[11]	WU Wei, LI Zeping, YANG Huawei, LIN Chuan, WANG Zhongde. Deep attention video popularity prediction model fusing content features and temporal information [J]. Journal of Computer Applications, 2021, 41(7): 1878-1884.
[12]	LI Yangzhi, YUAN Jiazheng, LIU Hongzhe. Human skeleton-based action recognition algorithm based on spatiotemporal attention graph convolutional network model [J]. Journal of Computer Applications, 2021, 41(7): 1915-1921.
[13]	ZHANG Yang, JIANG Minghu. Authorship identification of text based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(7): 1897-1901.
[14]	LI Chao, LAN Hai, WEI Xian. Attention-based object detection with millimeter wave radar-lidar fusion [J]. Journal of Computer Applications, 2021, 41(7): 2137-2144.
[15]	WU Lidan, XUE Yuyang, TONG Tong, DU Min, GAO Qinquan. Image colorization algorithm based on foreground semantic information [J]. Journal of Computer Applications, 2021, 41(7): 2048-2053.