Chromosome image segmentation framework based on improved Mask R-CNN

doi:10.11772/j.issn.1001-9081.2020030355

Abstract

Abstract: The manual segmentation of chromosome images is time-consuming and laborious, and the accuracy of current automatic segmentation methods is not poor. Therefore, based on improved Mask R-CNN (Mask Region-based Convolutional Neural Network), a chromosome image segmentation framework named Mask Oriented R-CNN (Mask Oriented Region-based Convolutional Neural Network) was proposed, which introduced orientation information to perform instance segmentation of chromosome images. Firstly, the regression branch of oriented bounding boxes was added to predict the compact bounding boxes and obtain orientation information. Secondly, a novel Intersection-over-Union (IoU) metric called AwIoU (Angle-weighted Intersection-over-Union) was proposed to improve the criterion of redundant bounding boxes by combining the relationship between the orientation information and edges. Finally, the oriented convolutional path structure was realized to reduce the interference in mask prediction by copying the path of mask branch and selecting the training path according to the orientation information of the instances. Experimental results show that compared with the baseline model Mask R-CNN, Mask Oriented R-CNN has the mean average precision increased by 10.22 percentage points when the IoU threshold is 0.5, and the mean metric increased by 4.91 percentage points when the IoU threshold is from 0.5 to 0.95. Experimental results show that the Mask Oriented R-CNN framework achieves better segmentation results than the baseline model in chromosome image segmentation, which is helpful to achieve automatic segmentation of chromosome images.

Key words: Convolutional Neural Network (CNN), instance segmentation, Mask R-CNN (Mask Region-based Convolutional Neural Network), chromosome image segmentation, image segmentation, Non-Maximum Suppression (NMS), Intersection-over-Union (IoU)

摘要： 针对染色体图像的人工分割耗时费力且当前自动分割方法精度不佳的问题，基于改进的Mask R-CNN提出了一种染色体图像分割框架——Mask Oriented R-CNN，引入方向信息对染色体图像进行实例分割。首先，新增有向包围框回归分支，以预测紧实包围框并获取方向信息；然后，提出新的交并比（IoU）度量——角度加权交并比（AwIoU），从而结合方向信息与边的关系以改进冗余包围框的判据；最后，实现有向卷积通路结构，通过拷贝掩模分支通路并依据实例的方向信息选择训练路径来减少掩模预测中的干扰。实验结果表明，相较于基准模型Mask R-CNN，Mask Oriented R-CNN在IoU阈值为0.5时的平均精度均值指标提升了10.22个百分点，IoU阈值为0.5~0.95时的平均指标提升了4.91个百分点。研究结果显示，Mask Oriented R-CNN框架相较于基准模型取得了更好的染色体图像分割结果，有助于实现染色体图像自动分割。

关键词: 卷积神经网络, 实例分割, Mask R-CNN, 染色体图像分割, 图像分割, 非极大值抑制, 交并比

CLC Number:

FENG Tao, CHEN Bin, ZHANG Yuefei. Chromosome image segmentation framework based on improved Mask R-CNN[J]. Journal of Computer Applications, 2020, 40(11): 3332-3339.

冯涛, 陈斌, 张跃飞. 基于改进的Mask R-CNN的染色体图像分割框架[J]. 计算机应用, 2020, 40(11): 3332-3339.

References

[1] 陆国辉, 徐湘民. 临床遗传咨询[M]. 北京:北京大学医学出版社,2007:38-48.(LU G H,XU X M. Clinical Genetic Counseling[M]. Beijing:Peking University Medical Press,2007:38-48.)
[2] 朱军. 出生缺陷及其监测[J]. 中国实用妇科与产科杂志,2002, 18(9):513-514.(ZHU J. Birth defects and their monitoring[J]. Chinese Journal of Practical Gynecology and Obstetrics,2002,18(9):513-514.)
[3] WILSON R D, BLIGHT C, LANGLOIS S. Diagnosing chromosomal abnormalities from"big"to"small"with molecular cytogenetic technology[J]. Journal of Obstetrics and Gynaecology Canada,2009,31(5):414-421.
[4] 闫文忠. 染色体自动分析系统的研究现状及未来目标[J]. 中国组织工程研究与临床康复,2009,13(13):2544-2546.(YAN W Z. Research status and future development of chromosome automatic analysis system[J]. Journal of Clinical Rehabilitative Tissue Engineering Research,2009,13(13):2544-2546.)
[5] HE K,GKIOXARI G,DOLLÁR P,et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:2980-2988.
[6] SHELHAMER E,LONG J,DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651.
[7] RONNEBERGER O,FISCHER P,BROX T. U-Net:convolutional networks for biomedical image segmentation[C]//Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS 9351. Cham:Springer, 2015:234-241.
[8] HU R L,KARNOWSKI J,FADELY R,et al. Image segmentation to distinguish between overlapping human chromosomes[EB/OL].[2019-11-26]. https://arxiv.org/pdf/1712.07639.pdf.
[9] SALEH H,SAAD N H,ISA N A M. Overlapping chromosome segmentation using U-Net:convolutional networks with test time augmentation[J]. Procedia Computer Science, 2019, 159:524-533.
[10] ALTINSOY E, YILMAZ C, WEN J, et al. Raw G-band chromosome image segmentation using U-Net based neural network[C]//Proceedings of the 2019 International Conference on Artificial Intelligence and Soft Computing,LNCS 11509. Cham:Springer,2019:117-126.
[11] PARDO E, MORGADO J M T, MALPICA N. Semantic segmentation of mFISH images using convolutional networks[J]. Cytometry,Part A:Journal of Quantitative Cell Science,2018,93(6):620-627.
[12] CHEN L C,PAPANDREOU G,KOKKINOS I,et al. DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[13] XIAO L, LUO C, YU T, et al. DeepACEv2:automated chromosome enumeration in metaphase cell images using deep convolutional neural networks[EB/OL].[2019-12-31]. https://arxiv.org/pdf/1910.11091.pdf.
[14] REN S,HE K,GIRSHICK R,et al. Faster R-CNN:towards realtime object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017, 39(6):1137-1149.
[15] XIE N,LI X,LI K,et al. Statistical karyotype analysis using CNN and geometric optimization[J]. IEEE Access,2019,7:179445-179453.
[16] GIRSHICK R,DONAHUE J,DARRELL T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2014:580-587.
[17] UIJLINGS J R R,VAN DE SANDE K E A,GEVERS T,et al. Selective search for object recognition[J]. International Journal of Computer Vision,2013,104(2):154-171.
[18] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:1440-1448.
[19] LIU S,QI L,QIN H,et al. Path aggregation network for instance segmentation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:8759-8768.
[20] HUANG Z,HUANG L,GONG Y,et al. Mask scoring R-CNN[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:6402-6411.
[21] MA J,SHAO W,YE H,et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE Transactions on Multimedia,2018,20(11):3111-3122.
[22] HE Y,ZHU C,WANG J,et al. Bounding box regression with uncertainty for accurate object detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:2883-2892.
[23] HUANG J, SIVAKUMAR V, MNATSAKANYAN M, et al. Improving rotated text detection with rotation region proposal networks[EB/OL].[2019-11-26]. https://arxiv.org/pdf/1811.07031.pdf.
[24] COCOdataset. Detection evaluation[EB/OL].[2019-11-26]. http://cocodataset.org/#detection-eval.
[25] Matterport,Inc.. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow[EB/OL].[2019-11-26]. https://github.com/matterport/Mask_RCNN.
[26] BODLA N,SINGH B,CHELLAPPA R,et al. Soft-NMS-improving object detection with one line of code[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:5562-5570.