Feature point localization of left ventricular ultrasound image based on convolutional neural network

doi:10.11772/j.issn.1001-9081.2018091931

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (4): 1201-1207.DOI: 10.11772/j.issn.1001-9081.2018091931

Previous Articles Next Articles

Feature point localization of left ventricular ultrasound image based on convolutional neural network

ZHOU Yujin^1,2, WANG Xiaodong¹, ZHANG Lige^1,2, ZHU Kai^1,2, YAO Yu¹

1. Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu Sichuan 610041, China;
2. University of Chinese Academy of Sciences, Beijing 100049, China

Received:2018-09-17 Revised:2018-10-10 Online:2019-04-10 Published:2019-04-10
Supported by:
This work is partially supported by the Key Research and Development Project of Science and Technology Department of Sichuan Province (2017SZ0010), the Science and Technology Support Project of Sichuan Province (2016JZ0035).

基于卷积神经网络的左心室超声图像特征点定位

周玉金^1,2, 王晓东¹, 张力戈^1,2, 朱锴^1,2, 姚宇¹

1. 中国科学院成都计算机应用研究所, 成都 610041;
2. 中国科学院大学, 北京 100049

通讯作者: 周玉金
作者简介:周玉金(1993-),女,云南曲靖人,硕士研究生,主要研究方向:深度学习、医学图像处理;王晓东(1973-),男,四川乐山人,研究员,主要研究方向:地理信息系统、物联网;张力戈(1995-),男,山西太原人,博士研究生,主要研究方向:优化算法、深度学习;朱锴(1991-),男,贵州安顺人,博士研究生,主要研究方向:机器学习、深度学习;姚宇(1980-),男,四川宜宾人,研究员,博士,主要研究方向:机器学习、数据挖掘。
基金资助:
四川省科技厅重点研发项目（2017SZ0010）；四川省科技支撑计划项目（2016JZ0035）。

Abstract

Abstract: In order to solve the problem that the traditional cascaded Convolutional Neural Network (CNN) has low accuracy of feature point localization in left ventricular ultrasound image, an improved cascaded CNN with region extracted by Faster Region-based CNN (Faster-RCNN) model was proposed to locate the left ventricular endocardial and epicardial feature points in ultrasound images. Firstly, the traditional cascaded CNN was improved by a structure of two-stage cascaded. In the first stage, an improved convolutional network was used to roughly locate the endocardial and epicardial joint feature points. In the second stage, four improved convolutional networks were used to fine-tune the endocardial feature points and the epicardial feature points separately. After that, the positions of joint contour feature points were output. Secondly, the improved cascaded CNN was merged with target region extraction, which means that the target region containing the left ventricle was extracted by the Faster-RCNN model and then was sent into the improved cascaded CNN. Finally, the left ventricular contour feature points were located from coarse to fine. Experimental results show that compared with the traditional cascaded CNN, the proposed method is much more accurate in left ventricle feature point localization, and its prediction points are closer to the actual values. Under the root mean square error evaluation standard, the accuracy of feature point localization is improved by 32.6 percentage points.

Key words: echocardiography, left ventricle, feature points location, Convolutional Neural Network (CNN), cascaded convolutional neural network

摘要： 针对传统级联卷积神经网络（CNN）在左心室超声图像中定位准确度较低的问题，提出一种融合更快速区域卷积神经网络（Faster-RCNN）模型提取区域的级联卷积神经网络，实现对超声图像中左心室心内膜和心外膜轮廓特征点的定位。首先，采用两级级联的方式改进传统级联卷积神经网络的网络结构，第一级网络利用一个改进的卷积网络粗略定位左心室心内膜和心外膜联合的特征点，第二级网络使用四个改进的卷积网络分别对心内膜特征点和心外膜特征点进行位置微调，之后定位输出左心室心内膜和心外膜联合的轮廓特征点位置；然后，将改进的级联卷积神经网络与目标区域提取融合，即利用Faster-RCNN模型提取包含左心室的目标区域并将目标区域送入改进的级联卷积神经网络；最后，由粗到细对左心室轮廓特征点进行定位。实验结果表明，与传统级联卷积神经网络相比，所提方法在左心室超声图像上的定位效果更好，更逼近真实值，在均方根误差的评价标准下，特征点定位准确度提升了32.6个百分点。

关键词: 超声心动图, 左心室, 特征点定位, 卷积神经网络, 级联卷积神经网络

CLC Number:

TP391.41

ZHOU Yujin, WANG Xiaodong, ZHANG Lige, ZHU Kai, YAO Yu. Feature point localization of left ventricular ultrasound image based on convolutional neural network[J]. Journal of Computer Applications, 2019, 39(4): 1201-1207.

周玉金, 王晓东, 张力戈, 朱锴, 姚宇. 基于卷积神经网络的左心室超声图像特征点定位[J]. 计算机应用, 2019, 39(4): 1201-1207.

References

[1] NOBLE J A, BOUKERROUI D. Ultrasound image segmentation:a survey[J]. IEEE Transactions on Medical Imaging, 2006, 25(8):987-1010.
[2] 冉隆科. 一种基于角点检测方法的骨龄图像关键点定位[J]. 电子设计工程, 2011, 19(14):175-177. (RAN L K. A corner detection method based on the image of the skeletal age point positioning[J]. Electronic Design Engineering, 2011, 19(14):175-177.)
[3] MAHAPATRA D. Landmark detection in cardiac MRI using learned local image statistics[C]//STACOM 2012:Proceedings of the 2012 Statistical Atlases and Computational Models of the Heart - Imaging and Modelling Challenges, LNCS 7746. Berlin:Springer, 2012:115-124.
[4] YANG X, JIN J, XU M, et al. Ultrasound common carotid artery segmentation based on active shape model[J]. Computational and Mathematical Methods in Medicine, 2013, 2013(2):345968.
[5] COOTES T F, TAYLOR C J, COOPER D H, et al. Active shape models-their training and application[J]. Computer Vision & Image Understanding, 1995, 61(1):38-59.
[6] VARGAS-QUINTERO L, ESCALANTE-RAMÍREZ B, MARÍN L C, et al. Left ventricle segmentation in fetal echocardiography using a multi-texture active appearance model based on the steered Hermite transform[J]. Computer Methods and Programs in Biomedicine, 2016, 137:231-245.
[7] COOTES T F, EDWARDS G J, TAYLOR C J. Active appearance models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6):681-685.
[8] DOLLÁR P, WELINDER P, PERONA P. Cascaded pose regression[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2010:1078-1085.
[9] SUN P, ZHOU H, LUNDINE D, et al. Fast segmentation of left ventricle in CT images by explicit shape regression using random pixel difference features[EB/OL].[2018-05-10]. https://arxiv.org/pdf/1507.07508.
[10] SEDAI S, ROY P, GARNAVI R. Segmentation of right ventricle in cardiac mr images using shape regression[C]//MLMI 2015:Proceedings of the 2015 Machine Learning in Medical Imaging. Berlin:Springer, 2015:1-8.
[11] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[12] PRASOON A, PETERSEN K, IGEL C, et al. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network[C]//MICCAI 2013:Proceedings of the 2013 Medical Image Computing and Computer-Assisted Intervention. Berlin:Springer, 2013:246-253.
[13] YANG D, ZHANG S, YAN Z, et al. Automated anatomical landmark detection ondistal femur surface using convolutional neural network[C]//Proceedings of the 12th IEEE International Symposium on Biomedical Imaging. Piscataway, NJ:IEEE, 2015:17-21.
[14] REN S, HE K, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[15] SUN Y, WANG X, TANG X. Deep convolutional network cascade for facial point detection[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2013:3476-3483.
[16] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[EB/OL].[2018-05-10]. http://www.nvidia.in/content/tesla/pdf/machine-learning/imagenet-classification-with-deep-convolutional-nn.pdf.
[17] NAIR V, HINTON G E. Rectified linear units improve restricted Boltzmann machines[C]//ICML 2010:Proceedings of the 27th International Conference on International Conference on Machine Learning. Madison, Wisconsin:Omnipress, 2010:807-814.
[18] GIRSHICK R, DONAHUE J, DARRELL T, et al. Region-based convolutional networks for accurate object detection and segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(1):142-158.
[19] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[C]//ECCV 2014:Proceedings of the 2014 European Conference on Computer Vision. Berlin:Springer, 2014:346-361.
[20] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1440-1448.
[21] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2015:3431-3440.
[22] 纪祥虎, 高思聪, 黄志标, 等. 基于Centripetal CatmullRom曲线的经食道超声心动图左心室分割方法[J]. 四川大学学报(工程科学版), 2016, 48(5):84-90. (JI X H, GAO S C, HUANG Z B, et al. Left ventricle segmentation in transesophageal echocardiography based on Centripetal CatmullRom curve[J]. Journal of Sichuan University (Engineering Science Edition), 2016, 48(5):84-90.)

Feature point localization of left ventricular ultrasound image based on convolutional neural network

基于卷积神经网络的左心室超声图像特征点定位

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[2]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[8]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[9]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[10]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[11]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.
[12]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[13]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[14]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[15]	YAN Junhua, HOU Ping, ZHANG Yin, LYU Xiangyang, MA Yue, WANG Gaofei. Image single distortion type judgment method based on two-channel convolutional neural network [J]. Journal of Computer Applications, 2021, 41(6): 1761-1766.