基于多尺度和跨空间融合的超声乳腺结节分割

doi:10.11772/j.issn.1001-9081.2022111673

《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (11): 3599-3606.DOI: 10.11772/j.issn.1001-9081.2022111673

• 多媒体计算与计算机仿真 • 上一篇

基于多尺度和跨空间融合的超声乳腺结节分割

赵欣¹(), 祝倩倩¹, 赵聪², 吴佳玲³

^1.大连大学信息工程学院, 辽宁大连 116622
^2.大连市第五人民医院超声科, 辽宁大连 116021
^3.大连医科大学附属第一医院超声科, 辽宁大连 116011

收稿日期:2022-11-11 修回日期:2023-02-16 接受日期:2023-02-20 发布日期:2023-03-10 出版日期:2023-11-10
通讯作者: 赵欣
作者简介:赵欣（1974—），女，辽宁锦州人，副教授，博士，CCF会员，主要研究方向：人工智能、数字医学图像处理 zhaoxin@dlu.edu.cn
祝倩倩（1995—），女，山西朔州人，硕士研究生，硕士，CCF会员，主要研究方向：医学图像处理
赵聪（1984—），女，辽宁大连人，副主任医师，主要研究方向：甲状腺、乳腺疾病的专科诊断及超声介入治疗
吴佳玲（1979—），女，辽宁大连人，副主任医师，硕士，主要研究方向：甲状腺、乳腺疾病的超声诊断。
基金资助:
国家自然科学基金资助项目(61971424)

Segmentation of breast nodules in ultrasound images based on multi-scale and cross-spatial fusion

Xin ZHAO¹(), Qianqian ZHU¹, Cong ZHAO², Jialing WU³

^1.School of Information Engineering，Dalian University，Dalian Liaoning 116622，China
^2.Department of Ultrasound，The Fifth People’s Hospital of Dalian，Dalian Liaoning 116021，China
^3.Department of Ultrasound，The First Affiliated Hospital of Dalian Medical University，Dalian Liaoning 116011，China

Received:2022-11-11 Revised:2023-02-16 Accepted:2023-02-20 Online:2023-03-10 Published:2023-11-10
Contact: Xin ZHAO
About author:ZHAO Xin， born in 1974， Ph. D.， associate professor. Her research interests include artificial intelligence， digital medical image processing.
ZHU Qianqian， born in 1995，M. S. candidate. Her research interests include medical image processing.
ZHAO Cong， born in 1984， deputy chief physician. Her research interests include specialized diagnosis and interventional ultrasound therapy of thyroid and breast diseases.
WU Jialing， born in 1979， M. S.， deputy chief physician. Her research interests include ultrasound diagnosis of breast and thyroid diseases.
Supported by:
National Natural Science Foundation of China(61971424)

摘要/Abstract

摘要：

针对超声成像分辨率低且存在噪声、结节形态与纹理复杂多变导致超声乳腺结节精确分割较为困难的问题，提出一种融合多尺度特征提取和跨空间特征融合的超声乳腺结节端到端自动分割方法。首先，设计一种多尺度特征提取与融合（MFEF）模块，通过融合4条具有不同感受野的卷积路径使网络具有多尺度的特征提取能力。其次，为对高级语义信息进行多尺度观察和信息筛选，在瓶颈层采用尺度感知与特征聚合（SFA）模块，以增强编码阶段的深层特征提取能力。此外，设计跨空间残差融合（CRF）模块，并将它应用在编、解码器间的跳跃连接上。该模块一方面对不同编码层进行跨空间信息融合，实现不同编码层间的信息互补；另一方面进一步提取编码层信息特征，缓解编解码对等层之间的语义差异，从而更好地补偿解码阶段的信息损失。在公开的超声乳腺结节数据集上的实验结果显示，所提方法的DICE系数可达0.888，同主流的深度学习分割模型UNet、AttUNet、ResUNet++、SKUNet相比，提高了0.033~0.094，对比相同数据集中的改进模型如CF²-Net、ESTAN、FS-UNet、SMU-Net，提高了0.001~0.068。所提方法分割结果图的主观视觉效果与专家给出的金标准最接近，能更加准确地分割出乳腺结节区域。

关键词: 多尺度特征融合, 跨空间特征融合, 尺度感知与特征聚合, 卷积神经网络, 超声乳腺结节分割

Abstract:

Accurate breast nodule segmentation in ultrasound images is very challenging due to the low resolution and noise of ultrasound imaging， as well as the complexity and variability of the shape and texture of nodules， therefore， an end-to-end automatic segmentation method of breast nodules in ultrasound images based on multi-scale and cross-spatial feature fusion was proposed. Firstly， a Multi-scale Feature Extraction and Fusion （MFEF） module was designed to enable the network to have multi-scale feature extraction ability by fusing four convolutional paths with different receptive fields. Then， for the multi-scale observation and information filtering of high-level semantic information， a Scale-aware Feature Aggregation （SFA） module was used at the bottleneck layer to enhance the deep feature extraction ability in the encoding stage. Besides， a Cross-spatial Residual Fusion （CRF） module was designed and applied to the skip connection between the encoder and decoder to fuse information among different encoding layers in a cross-spatial way and implement information complementarity between different encoding layers， further extract information features of encoding layer and narrow the difference between peer layers of encoder and decoder to better compensate for the information loss in the decoding stage. Experimental results on a public ultrasound breast nodule dataset show that the proposed method achieves DICE coefficient of 0.888， which is 0.033 to 0.094 higher than those of the mainstream deep learning segmentation models UNet， AttUNet， ResUNet++ and SKUNet， and is 0.001 to 0.068 higher than those of the improved models such as CF²-Net， Estan， FS-UNet and SMU-Net in the same dataset. The subjective visualization of the segmentation result of the proposed method is closest to the gold standards provided by experts， verifying that the proposed mehtod can segment the breast nodule area more accurately.

Key words: multi-scale feature fusion, cross-spatial feature fusion, Scale-aware Feature Aggregation (SFA), Convolutional Neural Network (CNN), breast nodule segmentation in ultrasound images

中图分类号:

TP 391

赵欣, 祝倩倩, 赵聪, 吴佳玲. 基于多尺度和跨空间融合的超声乳腺结节分割[J]. 计算机应用, 2023, 43(11): 3599-3606.

Xin ZHAO, Qianqian ZHU, Cong ZHAO, Jialing WU. Segmentation of breast nodules in ultrasound images based on multi-scale and cross-spatial fusion[J]. Journal of Computer Applications, 2023, 43(11): 3599-3606.

图/表 12

图1 乳腺超声图像

Fig. 1 Breast ultrasound images

图2 本文模型的网络结构

Fig. 2 Network structure of proposed model

图3 MFEF模块结构

Fig. 3 Structure of MFEF module

图4 SplitB与SplitB+的结构

Fig. 4 Structures of SplitB and SplitB+

图5 SFA模块

Fig. 5 SFA module

图6 CRF与DSF模块

Fig. 6 CRF and DSF modules

图7 超声乳腺结节图像示例

Fig. 7 Ultrasound image example of breast nodules

表1 网络结构消融性能对比

Tab. 1 Performance comparison of network structure ablation

组号	方法	DICE系数	召回率	精度	交并比
1	UNet	0.794	0.796	0.879	0.709
2	UNet+SplitB	0.816	0.821	0.880	0.732
2	UNet+SplitB+	0.827	0.835	0.885	0.749
3	UNet+MFEF	0.847	0.852	0.896	0.762
	UNet+SFA	0.833	0.805	0.933	0.754
	UNet+CRF	0.825	0.794	0.936	0.744
4	UNet+MFEF+SFA	0.875	0.867	0.906	0.790
	UNet+SFA+CRF	0.854	0.828	0.928	0.778
	UNet+MFEF+CRF	0.858	0.839	0.930	0.784
5	UNet+MFEF+SFA+CRF	0.888	0.875	0.916	0.808

图8 消融实验分割结果对比

Fig. 8 Segmentation results of ablation experiments

图9 本文模型在公开数据集上的Loss变化

Fig. 9 Loss change of proposed model on public dataset

表2 不同分割方法的比较

Tab. 2 Comparison of different segmentation methods

方法	DICE系数	召回率	精度	交并比
UNet	0.794	0.796	0.879	0.709
AttUNet	0.817	0.809	0.902	0.724
ResUNet++	0.849	0.858	0.867	0.756
SKUNet	0.855	0.847	0.907	0.773
CF²-Net	0.856	0.852	0.882	—
ESTAN	0.820	0.840	—	0.740
FS-UNet	0.887	0.864	—	0.804
SMU-Net	0.870	0.889	0.880	—
本文方法	0.888	0.875	0.916	0.808

图10 不同模型的分割结果

Fig. 10 Segmentation results of different models

参考文献 28

1	司爽，张伟，王慧颖，等. 乳腺X线摄影、超声及磁共振诊断乳腺癌的性能评价［J］. 中国临床医学影像杂志， 2020， 31（1）：20-23，29. 10.12117/jccmi.2020.01.005
	SI S， ZHANG W， WANG H Y， et al. Performance evaluation of breast cancer diagnosed with mammography， ultrasound and MRI［J］. Journal of China Clinical Medical Imaging， 2020， 31（1）： 20-23， 29. 10.12117/jccmi.2020.01.005
2	JIANG P， PENG J， ZHANG G， et al. Learning-based automatic breast tumor detection and segmentation in ultrasound images［C］// Proceedings of the 9th IEEE International Symposium on Biomedical Imaging. Piscataway： IEEE， 2012： 1587-1590. 10.1109/isbi.2012.6235878
3	SHAN J， CHENG H D， WANG Y. A novel segmentation method for breast ultrasound images based on neutrosophic l-means clustering［J］. Medical Physics， 2012， 39（9）： 5669-5682. 10.1118/1.4747271
4	LUO Y， HAN S， HUANG Q. A novel graph-based segmentation method for breast ultrasound images［C］// Proceedings of the 2016 International Conference on Digital Image Computing： Techniques and Applications. Piscataway： IEEE， 2016： 1-6. 10.1109/dicta.2016.7796992
5	LIU L， LI K， QIN W， et al. Automated breast tumor detection and segmentation with a novel computational framework of whole ultrasound images［J］. Medical and Biological Engineering and Computing， 2018， 56（2）： 183-199. 10.1007/s11517-017-1770-3
6	LOTFOLLAHI M， GITY M， YE J Y， et al. Segmentation of breast ultrasound images based on active contours using neutrosophic theory［J］. Journal of Medical Ultrasonics， 2018， 45（2）： 205-212. 10.1007/s10396-017-0811-8
7	ZHUANG Z， LI N， JOSEPH RAJ A N， et al. An RDAU-NET model for lesion segmentation in breast ultrasound images［J］. PLoS ONE， 2019， 14（8）： No.e0221535. 10.1371/journal.pone.0221535
8	HAN L， HUANG Y， DOU H， et al. Semi-supervised segmentation of lesion from breast ultrasound images with attentional generative adversarial network［J］. Computer Methods and Programs in Biomedicine， 2020， 189： No.105275. 10.1016/j.cmpb.2019.105275
9	BYRA M， JAROSIK P， SZUBERT A， et al. Breast mass segmentation in ultrasound with selective kernel U-Net convolutional neural network［J］. Biomedical Signal Processing and Control， 2020， 61： No.102027. 10.1016/j.bspc.2020.102027
10	VAKANSKI A， XIAN M， FREER P E. Attention-enriched deep learning model for breast tumor segmentation in ultrasound images［J］. Ultrasound in Medicine and Biology， 2020， 46（10）： 2819-2833. 10.1016/j.ultrasmedbio.2020.06.015
11	XUE C， ZHU L， FU H， et al. Global guidance network for breast lesion segmentation in ultrasound images［J］. Medical Image Analysis， 2021， 70： No.101989. 10.1016/j.media.2021.101989
12	LI J， CHENG L， XIA T， et al. Multi-scale fusion U-Net for the segmentation of breast lesions［J］. IEEE Access， 2021， 9： 137125-137139. 10.1109/access.2021.3117578
13	GAO C， YE H， CAO F， et al. Multiscale fused network with additive channel-spatial attention for image segmentation［J］. Knowledge-Based Systems， 2021， 214： No.106754. 10.1016/j.knosys.2021.106754
14	IQBAL A， SHARIF M. MDA-Net： multiscale dual attention-based network for breast lesion segmentation using ultrasound images［J］. Journal of King Saud University — Computer and Information Sciences， 2022， 34（9）： 7283-7299. 10.1016/j.jksuci.2021.10.002
15	PUNN N S， AGARWAL S. RCA-IUnet： a residual cross-spatial attention-guided inception U-Net model for tumor segmentation in breast ultrasound imaging［J］. Machine Vision and Applications， 2022， 33（2）： No.27. 10.1007/s00138-022-01280-3
16	CHEN G， LI L， DAI Y， et al. AAU-Net： an adaptive attention U-Net for breast lesions segmentation in ultrasound images［J］. IEEE Transactions on Medical Imaging， 2023， 42（5）： 1289-1300. 10.1109/tmi.2022.3226268
17	ZHAO T， DAI H. Breast tumor ultrasound image segmentation method based on improved residual U-Net network［J］. Computational Intelligence and Neuroscience， 2022， 2022： No.3905998. 10.1155/2022/3905998
18	大连大学. 基于多尺度和跨空间融合的超声乳腺结节端到端分割方法： 202211177184.X［P］. 2023-01-06.
	Dalian University. End-to-end segmentation of ultrasound breast nodules based on multi-scale and cross-spatial fusion： 202211177184.X［P］. 2023-01-06.
19	YUAN P， LIN S， CUI C， et al. HS-ResNet： hierarchical-split block on convolutional neural network［EB/OL］. （2020-10-15）［2022-12-15］.. 10.48550/arXiv.2010.07621
20	WU H， WANG W， ZHONG J， et al. SCS-Net： a scale and context sensitive network for retinal vessel segmentation［J］. Medical Image Analysis， 2021， 70： No.102025. 10.1016/j.media.2021.102025
21	YAP M H， PONS G， MARTÍ J， et al. Automated breast ultrasound lesions detection using convolutional neural networks［J］. IEEE Journal of Biomedical and Health Informatics， 2018， 22（4）： 1218-1226. 10.1109/jbhi.2017.2731873
22	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
23	OKTAY O， SCHLEMPER J， LE FOLGOC L， et al. Attention U-Net： learning where to look for the pancreas［EB/OL］. ［2022-10-12］..
24	JHA D， SMEDSRUD P H， RIEGLER M A， et al. ResUNet++： an advanced architecture for medical image segmentation［C］// Proceedings of the 2019 IEEE International Symposium on Multimedia. Piscataway： IEEE， 2019： 225-230. 10.1109/ism46123.2019.00049
25	NING Z， WANG K， ZHONG S， et al. CF²-Net： coarse-to-fine fusion convolutional network for breast ultrasound image segmentation［EB/OL］. （2020-03-23）［2022-10-13］.. 10.48550/arXiv.2003.10144
26	SHAREEF B， VAKANSKI A， FREER P E， et al. ESTAN： enhanced small tumor-aware network for breast ultrasound image segmentation［J］. Healthcare， 2022， 10（11）： No.2262. 10.3390/healthcare10112262
27	PI J， QI Y， LOU M， et al. FS-UNet： mass segmentation in mammograms using an encoder-decoder architecture with feature strengthening［J］. Computers in Biology and Medicine， 2021， 137： No.104800. 10.1016/j.compbiomed.2021.104800
28	NING Z， ZHONG S， FENG Q， et al. SMU-Net： saliency-guided morphology-aware U-Net for breast lesion segmentation in ultrasound image［J］. IEEE Transactions on Medical Imaging， 2022， 41（2）： 476-490. 10.1109/tmi.2021.3116087

[1]	路琨婷, 费蓉蓉, 张选德. 融合卷积神经网络的遥感图像全色锐化[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2963-2969.
[2]	尚绍法, 蒋林, 李远成, 朱筠. 异构平台下卷积神经网络推理模型自适应划分和调度方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2828-2835.
[3]	杨昊, 张轶. 基于上下文信息和多尺度融合重要性感知的特征金字塔网络算法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2727-2734.
[4]	李豆豆, 李汪根, 夏义春, 束阳, 高坤. 基于特征交互与自适应融合的骨骼动作识别[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2581-2587.
[5]	郑帅, 张晓龙, 邓鹤, 任宏伟. 基于多尺度特征融合和网格注意力机制的三维肝脏影像分割方法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2303-2310.
[6]	秦源源, 张鸿. 基于注意力特征金字塔网络的肺结节检测算法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2311-2318.
[7]	何嘉明, 杨巨成, 吴超, 闫潇宁, 许能华. 基于多模态图卷积神经网络的行人重识别方法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2182-2189.
[8]	张慧斌, 冯丽萍, 郝耀军, 王一宁. 基于注意力机制和迁移学习的古壁画朝代识别[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1826-1832.
[9]	杨森淇, 段旭良, 肖展, 郎松松, 李志勇. 基于ERNIE+DPCNN+BiGRU的农业新闻文本分类[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1461-1466.
[10]	许睿, 梁爽, 万航, 文益民, 沈世铭, 李建. 基于烛台图模式匹配的PM_2.5扩散特征的提取[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1394-1400.
[11]	隋佳宏, 毛莺池, 于慧敏, 王子成, 平萍. 基于图注意力网络的全局图像描述生成方法[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1409-1415.
[12]	何建辉, 胡春龙, 束鑫. 基于多峰标签分布学习的多任务年龄估计方法[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1578-1583.
[13]	傅励瑶, 尹梦晓, 杨锋. 基于Transformer的U型医学图像分割网络综述[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1584-1595.
[14]	王彬, 向甜, 吕艺东, 王晓帆. 基于NSGA‑Ⅱ的自适应多尺度特征通道分组优化算法[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1401-1408.
[15]	樊小宇, 蔺素珍, 王彦博, 刘峰, 李大威. 基于残差图卷积神经网络的高倍欠采样核磁共振图像重建算法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1261-1268.

基于多尺度和跨空间融合的超声乳腺结节分割

Segmentation of breast nodules in ultrasound images based on multi-scale and cross-spatial fusion

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 28

相关文章 15

编辑推荐

Metrics