Stomach cancer image segmentation method based on EfficientNetV2 and object-contextual representation

doi:10.11772/j.issn.1001-9081.2022081159

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (9): 2955-2962.DOI: 10.11772/j.issn.1001-9081.2022081159

• Multimedia computing and computer simulation • Previous Articles Next Articles

Stomach cancer image segmentation method based on EfficientNetV2 and object-contextual representation

Di ZHOU¹^,², Zili ZHANG¹^,²(), Jia CHEN¹^,³, Xinrong HU²^,³, Ruhan HE²^,³, Jun ZHANG⁴

^1.School of Computer Science and Artificial Intelligence，Wuhan Textile University，Wuhan Hubei 430200，China
^2.Engineering Research Center of Hubei Province for Clothing Information，Wuhan Textile University，Wuhan Hubei 430200，China
^3.Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion，Wuhan Textile University，Wuhan Hubei 430200，China
^4.School of Computer Science and Engineering，Wuhan Institute of Technology，Wuhan Hubei 430205，China

Received:2022-08-07 Revised:2022-11-03 Accepted:2022-11-14 Online:2023-01-11 Published:2023-09-10
Contact: Zili ZHANG
About author:ZHOU Di， born in 1997， M. S. candidate， His research interests include machine learning， image processing.
CHEN Jia， born in 1982， Ph. D.， lecturer. Her research interests include image processing， pattern recognition.
HU Xinrong， born in 1973， Ph. D.， professor. Her research interests include virtual reality， machine learning.
HE Ruhan， born in 1974， Ph. D.， professor. His research interests include artificial intelligence， computer vision.
ZHANG Jun， born in 1975， Ph. D.， professor. His research interests include machine learning， intelligent manufacturing.
Supported by:
Science and Technology Research Program of Department of Education of Hubei Province(B2017066)

基于EfficientNetV2和物体上下文表示的胃癌图像分割方法

周迪¹^,², 张自力¹^,²(), 陈佳¹^,³, 胡新荣²^,³, 何儒汉²^,³, 张俊⁴

^1.武汉纺织大学计算机与人工智能学院, 武汉 430200
^2.武汉纺织大学湖北省服装信息化工程技术研究中心, 武汉 430200
^3.武汉纺织大学纺织服装智能化湖北省工程研究中心, 武汉 430200
^4.武汉工程大学计算机科学与工程学院, 武汉 430205

通讯作者: 张自力
作者简介:周迪（1997—），男，湖北武汉人，硕士研究生，CCF会员，主要研究方向：机器学习、图像处理
陈佳（1982—），女，湖北武汉人，讲师，博士，CCF会员，主要研究方向：图像处理、模式识别
胡新荣（1973—），女，湖北武汉人，教授，博士，CCF会员，主要研究方向：虚拟现实、机器学习
何儒汉（1974—），男，湖北宜昌人，教授，博士，CCF会员，主要研究方向：人工智能、计算机视觉；
张俊（1975—），男，湖北随州人，教授，博士，CCF会员，主要研究方向：机器学习、智能制造
基金资助:
湖北省教育厅科学技术研究计划项目(B2017066)

Abstract

Abstract:

In view of the problems that the upsampling process of U-Net is easy to lose details， and the datasets of stomach cancer pathological image are generally small， which tends to lead to over-fitting， an automatic segmentation model for pathological images of stomach cancer based on improved U-Net was proposed， namely EOU-Net. In EOU-Net， based on the existing U-Net model， EfficientNetV2 was used as the backbone， thereby enhancing the feature extraction ability of the network encoder. In the decoding stage， the relations between cell pixels were explored on the basis of Object-Contextual Representation （OCR）， and the improved OCR module was used to solve the loss problem of the upsampled image details. Then， the post-processing of Test Time Augmentation （TTA） was used to predict the images obtained by rollover and rotations at different angles of the input image respectively， and then the prediction results of these images were combined by feature fusion to further optimize the output results of the network， thereby solving the problem of small medical datasets effectively. Experimental results on datasets SEED， BOT and PASCAL VOC 2012 show that the Mean Intersection over Union （MIoU） of EOU-Net is improved by 1.8， 0.6 and 4.5 percentage points respectively compared with that of OCRNet. It can be seen that EOU-Net can obtain more accurate segmentation results of stomach cancer images.

Key words: semantic segmentation, U-Net, EfficientNetV2, Object-Contextual Representation (OCR), stomach cancer

摘要：

针对U-Net上采样过程容易丢失细节信息，以及胃癌病理图像数据集普遍偏小，容易出现过拟合的问题，提出一种基于改进U-Net的自动分割胃癌病理图像模型EOU-Net。EOU-Net在U-Net模型的基础上，将EfficientNetV2作为骨干特征提取网络，以增强网络编码器的特征提取能力。在解码阶段，基于物体上下文表示（OCR）探究细胞像素间的关系，并使用改进后的OCR模块解决上采样图像的细节丢失问题；然后，使用验证阶段增强（TTA）后处理对输入图像进行翻转和不同角度旋转后分别预测这些图像，再通过特征融合的方式将多个输入图像预测结果进行合并，进一步优化网络的输出结果，从而有效解决医学数据集较小的问题。在SEED、BOT以及PASCAL VOC 2012数据集上的实验结果表明，与OCRNet相比，EOU-Net的平均交并比（MIoU）分别提高了1.8、0.6以及4.5个百分点。可见EOU-Net能得到更准确的胃癌图像分割结果。

关键词: 语义分割, U-Net, EfficientNetV2, 物体上下文表示, 胃癌

CLC Number:

TP391

Di ZHOU, Zili ZHANG, Jia CHEN, Xinrong HU, Ruhan HE, Jun ZHANG. Stomach cancer image segmentation method based on EfficientNetV2 and object-contextual representation[J]. Journal of Computer Applications, 2023, 43(9): 2955-2962.

周迪, 张自力, 陈佳, 胡新荣, 何儒汉, 张俊. 基于EfficientNetV2和物体上下文表示的胃癌图像分割方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2955-2962.

Figures/Tables 14

References 29

1	SUNG H， FERLAY J， SIEGEL R L， et al. Global cancer statistics 2020： GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries［J］. CA： A Cancer Journal for Clinicians， 2021， 71（3）： 209-249. 10.3322/caac.21660
2	HOOI J K Y， LAI W Y， NG W K， et al. Global prevalence of Helicobacter pylori infection： systematic review and meta-analysis［J］. Gastroenterology， 2017， 153（2）： 420-429. 10.1053/j.gastro.2017.04.022
3	许燕，汤烨，闫雯，等. 病理人工智能的现状和展望［J］. 中华病理学杂志， 2017， 46（9）： 593-595. 10.3760/cma.j.issn.0529-5807.2017.09.001
	XU Y， TANG Y， YAN W， et al. Present situation and prospect of pathological artificial intelligence［J］. Chinese Journal of Pathology， 2017， 46（9）： 593-595. 10.3760/cma.j.issn.0529-5807.2017.09.001
4	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 3431-3440. 10.1109/cvpr.2015.7298965
5	LIU Z W， LI X X， LUO P， et al. Semantic image segmentation via deep parsing network［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1377-1385. 10.1109/iccv.2015.162
6	ZHENG S， JAYASUMANA S， ROMERA-PAREDES B， et al. Conditional random fields as recurrent neural networks［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1529-1537. 10.1109/iccv.2015.179
7	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs［EB/OL］. （2016-06-07）［2022-09-16］.. 10.1109/tpami.2017.2699184
8	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. DeepLab： Semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（4）： 834-848. 10.1109/tpami.2017.2699184
9	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation［EB/OL］. （2017-12-05）［2022-09-16］.. 10.1007/978-3-030-01234-2_49
10	CHEN L C， ZHU Y K， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 833-851. 10.1007/978-3-030-01234-2_49
11	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
12	MILLETARI F， NAVAB N， AHMADI S A. V-Net： fully convolutional neural networks for volumetric medical image segmentation［C］// Proceedings of the 2016 International Conference on 3D Vision. Piscataway： IEEE， 2016： 565-571. 10.1109/3dv.2016.79
13	ALOM M Z， HASAN M， YAKOPCIC C， et al. Recurrent residual convolutional neural network based on U-Net （R2U-Net） for medical image segmentation［EB/OL］. （2018-05-29）［2022-09-16］.. 10.1109/naecon.2018.8556686
14	ZHOU Z W， SIDDIQUEE M M R， TAJBAKHSH N， et al. UNet++： redesigning skip connections to exploit multiscale features in image segmentation［J］. IEEE Transactions on Medical Imaging， 2020， 39（6）： 1856-1867. 10.1109/tmi.2019.2959609
15	OKTAY O， SCHLEMPER J， FOLGOC L L， et al. Attention U-Net： learning where to look for the pancreas［EB/OL］. （2018-05-20）［2022-09-16］..
16	张泽中，高敬阳，赵地. MIFNet：基于多尺度输入与特征融合的胃癌病理图像分割方法［J］. 计算机应用， 2019， 39（S2）： 107-113.
	ZHANG Z Z， GAO J Y， ZHAO D. MIFNet： pathological image segmentation method for stomach cancer based on multi-scale input and feature fusion［J］. Journal of Computer Applications， 2019， 39（S2）：107-113.
17	TAN M X， LE Q V. EfficientNetV2： smaller models and faster training［C］// Proceedings of the 38th International Conference on Machine Learning. New York： JMLR.org， 2021： 10096-10106.
18	TAN M X， LE Q V. EfficientNet： rethinking model scaling for convolutional neural networks［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 6105-6114.
19	ZOPH B， LE Q V. Neural architecture search with reinforcement learning［EB/OL］. （2017-02-15）［2022-09-16］..
20	HOWARD A G， ZHU M L， CHEN B， et al. MobileNets： efficient convolutional neural networks for mobile vision applications［EB/OL］. （2017-04-17）［2022-09-16］.. 10.48550/arXiv.1704.04861
21	ZHAO H S， SHI J P， QI X J， et al. Pyramid scene parsing network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6230-6239. 10.1109/cvpr.2017.660
22	WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7794-7803. 10.1109/cvpr.2018.00813
23	HUANG Z L， WANG X G， HUANG L C， et al. CCNet： criss-cross attention for semantic segmentation［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 603-612. 10.1109/iccv.2019.00069
24	VALMADRE J， BERTINETTO L， HENRIQUES J， et al. End-to-end representation learning for correlation filter based tracking［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5000-5008. 10.1109/cvpr.2017.531
25	CHEN W L， ZHU X G， SUN R Q， et al. Tensor low-rank reconstruction for semantic segmentation［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12362. Cham： Springer， 2020： 52-69.
26	YUAN Y H， CHEN X L， WANG J D. Object-contextual representations for semantic segmentation［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12351. Cham： Springer， 2020： 173-190.
27	ODENA A， DUMOULIN V， OLAH C. Deconvolution and checkerboard artifacts［J］. Distill， 2016， 2016： No.00003. 10.23915/distill.00003
28	WACHINGER C， REUTER M， KLEIN T. DeepNAT： deep convolutional neural network for segmenting neuroanatomy［J］. NeuroImage， 2018， 170： 434-445. 10.1016/j.neuroimage.2017.02.035
29	石志良，范伟楠，甘梓博，等. 骨边界增强滤波的图割算法［J/OL］. 南京师大学报（自然科学版）： 1-13 （2022-10-28）［2022-11-01］..
	SHI Z L， FAN W N， GAN Z B， et al. Graph-cut algorithm for bone boundary enhancement filtering［J/OL］. Journal of Nanjing Normal University（Natural Science Edition）： 1-13 （2022-10-28）［2022-11-03］..

Stage	Operator	编码器是否输出特征图
0	Conv3×3	􀳫
1	Fused-MBConv1，k 3×3	—
2	Fused-MBConv4，k 3×3	􀳫
3	Fused-MBConv4，k 3×3	􀳫
4	MBConv4，k 3×3，SE 0.25	􀳫
5	MBConv6，k 3×3，SE 0.25	—
6	MBConv6，k 3×3，SE 0.25	􀳫

Stage	Operator	编码器是否输出特征图
0	Conv3×3	􀳫
1	Fused-MBConv1，k 3×3	—
2	Fused-MBConv4，k 3×3	􀳫
3	Fused-MBConv4，k 3×3	􀳫
4	MBConv4，k 3×3，SE 0.25	􀳫
5	MBConv6，k 3×3，SE 0.25	—
6	MBConv6，k 3×3，SE 0.25	􀳫

方法	图像增强	特征融合	MIoU
TTA	垂直翻转	平均	81.20
		几何平均	79.50
		相加	81.20
	水平翻转	平均	81.20
		几何平均	79.50
		相加	81.20
	水平垂直翻转	平均	81.30
		几何平均	78.00
		相加	81.30
	水平垂直翻转+旋转	平均	81.40
		几何平均	76.70
		相加	81.40
DenseCRF-3	—	—	80.53
DenseCRF-5	—	—	80.49
DenseCRF-7	—	—	80.45

方法	图像增强	特征融合	MIoU
TTA	垂直翻转	平均	81.20
		几何平均	79.50
		相加	81.20
	水平翻转	平均	81.20
		几何平均	79.50
		相加	81.20
	水平垂直翻转	平均	81.30
		几何平均	78.00
		相加	81.30
	水平垂直翻转+旋转	平均	81.40
		几何平均	76.70
		相加	81.40
DenseCRF-3	—	—	80.53
DenseCRF-5	—	—	80.49
DenseCRF-7	—	—	80.45

数据集	模型	MIoU	不同种类的IoU
数据集	模型	MIoU	正常	病变
SEED	Att R2U-Net^*	71.2	72.1	70.3
	Att U-Net^*	74.3	76.5	72.2
	EOU-Net^*	76.5	78.3	74.9
	U-Net	80.1	81.4	78.9
	U-Net++	78.2	79.2	77.2
	DeepLabV3+^［10］	79.7	81.2	78.2
	OCRNet^［26］	79.6	80.8	78.5
	EOU-Net	81.4	82.5	80.3
BOT	Att R2U-Net^*	61.8	88.7	34.9
	Att U-Net^*	67.3	88.5	46.0
	EOU-Net^*	68.5	89.2	47.8
	U-Net	73.0	90.7	55.3
	U-Net++	72.8	90.5	55.1
	DeepLabV3+^［10］	73.1	90.3	55.9
	OCRNet^［26］	74.8	91.1	58.5
	EOU-Net	75.4	91.4	59.4

Stomach cancer image segmentation method based on EfficientNetV2 and object-contextual representation

基于EfficientNetV2和物体上下文表示的胃癌图像分割方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 29

Related Articles 15

Recommended Articles

Metrics

EfficientNetV2	OCR	TTA	MIoU
—	—	—	80.1
􀳫	—	—	80.5
􀳫	􀳫	—	80.8
􀳫	􀳫	􀳫	81.4

模型	MIoU	模型	MIoU
U-Net	46.5	DeepLabV3+^［10］	67.4
FCN^［4］	62.7	OCRNet^［26］	72.3
PSPNet^［24］	66.8	EOU-Net	76.8

[1]	Minghao SUN, Han YU, Yuqing CHEN, Kai LU. First-arrival picking and inversion of seismic waveforms based on U-shaped multilayer perceptron network [J]. Journal of Computer Applications, 2024, 44(7): 2301-2309.
[2]	Lijun XU, Hui LI, Zuyang LIU, Kansong CHEN, Weixuan MA. 3D-GA-Unet： MRI image segmentation algorithm for glioma based on 3D-Ghost CNN [J]. Journal of Computer Applications, 2024, 44(4): 1294-1302.
[3]	Wei LI, Ling CHEN, Xiuyuan XU, Min ZHU, Jixiang GUO, Kai ZHOU, Hao NIU, Yuchen ZHANG, Shanye YI, Yi ZHANG, Fengming LUO. Interstitial lung disease segmentation algorithm based on multi-task learning [J]. Journal of Computer Applications, 2024, 44(4): 1285-1293.
[4]	Pengfei ZHANG, Litao HAN, Hengjian FENG, Hongmei LI. Point cloud semantic segmentation based on attention mechanism and global feature optimization [J]. Journal of Computer Applications, 2024, 44(4): 1086-1092.
[5]	Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316.
[6]	Ning WU, Yangyang LUO, Huajie XU. Semantic segmentation method for remote sensing images based on multi-scale feature fusion [J]. Journal of Computer Applications, 2024, 44(3): 737-744.
[7]	Ziyi LI, Tingting QU, Qianpeng CHONG, Jindong XU. Remote sensing image segmentation network based on fuzzy multiscale features [J]. Journal of Computer Applications, 2024, 44(11): 3581-3586.
[8]	Yongjiang LIU, Bin CHEN. Pixel-level unsupervised industrial anomaly detection based on multi-scale memory bank [J]. Journal of Computer Applications, 2024, 44(11): 3587-3594.
[9]	Yusheng LIU, Xuezhong XIAO. High-fidelity image editing based on fine-tuning of diffusion model [J]. Journal of Computer Applications, 2024, 44(11): 3574-3580.
[10]	Qiumei ZHENG, Weiwei NIU, Fenghua WANG, Dan ZHAO. Dual-branch real-time semantic segmentation network based on detail enhancement [J]. Journal of Computer Applications, 2024, 44(10): 3058-3066.
[11]	Shuai ZHENG, Xiaolong ZHANG, He DENG, Hongwei REN. 3D liver image segmentation method based on multi-scale feature fusion and grid attention mechanism [J]. Journal of Computer Applications, 2023, 43(7): 2303-2310.
[12]	Bin LU, Jielin LIU. Semantic segmentation for 3D point clouds based on feature enhancement [J]. Journal of Computer Applications, 2023, 43(6): 1818-1825.
[13]	Liyao FU, Mengxiao YIN, Feng YANG. Transformer based U-shaped medical image segmentation network： a survey [J]. Journal of Computer Applications, 2023, 43(5): 1584-1595.
[14]	Quan YUAN, Yunpeng XU, Chengliang TANG. Document-level relation extraction method based on path labels [J]. Journal of Computer Applications, 2023, 43(4): 1029-1035.
[15]	You YANG, Ruhui ZHANG, Pengcheng XU, Kang KANG, Hao ZHAI. Improved U-Net for seal segmentation of Republican archives [J]. Journal of Computer Applications, 2023, 43(3): 943-948.