Universal vector flow mapping method combined with deep learning

doi:10.11772/j.issn.1001-9081.2021010045

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3368-3375.DOI: 10.11772/j.issn.1001-9081.2021010045

• Multimedia computing and computer simulation • Previous Articles Next Articles

Universal vector flow mapping method combined with deep learning

Bo PENG¹^,²^,³, Yaru LUO¹, Shenghua XIE²^,³(), Lixue YIN²^,³

^1.School of Computer Science，Southwest Petroleum University，Chengdu Sichuan 610500，China
^2.Cardiovascular Ultrasound and Non-invasive Cardiology Department，Sichuan Academy of Medical Sciences·Sichuan Provincial People’s Hospital，Chengdu Sichuan 610072，China
^3.Ultrasound in Cardiac Electrophysiology and Biomechanics Key Laboratory of Sichuan Province，Chengdu Sichuan 610072，China

Received:2021-01-12 Revised:2021-03-08 Accepted:2021-03-19 Online:2021-03-29 Published:2021-11-10
Contact: Shenghua XIE
About author:PENG Bo，born in 1980，Ph. D.，associate professor. His research interests include medical ultrasound imaging，medical image and signal analysis
LUO Yaru，born in 1995，M. S. candidate. Her research interests include medical image processing and analysis
XIE Shenghua，born in 1978，Ph. D.，associate research fellow. His research interests include medical image processing，cardiovascular biomechanics
YIN Lixue，born in 1964，M. S.，professor. His research interests include cardiovascular ultrasound.
Supported by:
the Application Foundation Research Project of Science and Technology Department of Sichuan Province(2018JY0649);the International Cooperation Project of Chengdu Science and Technology Bureau, Sichuan Province(2019-GH02-00040-HZ)

联合深度学习的通用血流向量成像方法

彭博¹^,²^,³, 罗娅茹¹, 谢盛华²^,³(), 尹立雪²^,³

^1.西南石油大学计算机科学学院，成都 610500
^2.四川省医学科学院·四川省人民医院心血管超声及心功能科，成都 610072
^3.超声心脏电生理学与生物力学四川省重点实验室，成都 610072

通讯作者: 谢盛华
作者简介:彭博（1980—），男，四川南充人，副教授，博士，CCF 会员，主要研究方向：医学超声成像、医学图像与信号分析
罗娅茹（1995—），女，四川绵阳人，硕士研究生，主要研究方向：医学图像处理与分析
谢盛华（1978—），男，重庆人，副研究员，博士，主要研究方向：医学图像处理、心血管生物力学
尹立雪（1964—），男，四川眉山人，教授，硕士，主要研究方向：心血管超声。
基金资助:
四川省科技厅应用基础研究项目(2018JY0649);四川省成都市科技局国际合作项目(2019-GH02-00040-HZ)

Abstract

Abstract:

The traditional ultrasound Vector Flow Mapping （VFM） technology has the limitation that it requires the proprietary software to obtain raw Doppler and speckle tracking data. In order to solve the problem， a universal VFM method combined with deep learning was proposed. Firstly， the velocity scale was used to obtain the velocities along the acoustic beam direction provided by the color Doppler echocardiogram as the radial velocity components. Then， the U-Net model was used to automatically identify the contour of the left ventricular wall， the left ventricular wall velocities were calculated by the retrained CNNs for optical flow using Pyramid， Warping， and Cost volume （PWC-Net） model as the boundary condition of the continuity equation， and the velocity component of each blood particle perpendicular to the acoustic beam direction （that was the tangential velocity component） was obtained by solving the continuity equation. Finally， the velocity vector map of the heart flow field was synthesized， and the visualization of the streamline chart of the heart flow field was realized. Experimental results show that， the velocity vector map and streamline chart of the heart flow field obtained by the proposed method can accurately reflect the corresponding time phases of left ventricular， the obtained visualized results are consistent with the analysis results of the VFM workstation provided by Aloka， and conform to the characteristics of left ventricular fluid dynamics. As a universal and fast VFM method， the proposed method do not need any vendor’s technical support and proprietary software， and can further promote the application of VFM in clinical workflow.

Key words: left ventricular, color Doppler, wall motion, fluid movement, visualization

摘要：

针对传统的超声血流向量成像（VFM）技术需要专有软件来获取原始多普勒和散斑跟踪数据的限制，提出一种联合深度学习的通用VFM方法。首先，使用速度标尺获取彩色多普勒超声心动图提供的沿声束方向的速度作为径向速度分量；然后，使用U-Net模型自动识别左心室壁轮廓，通过重新训练的PWC-Net模型计算左心室壁速度作为连续性方程的边界条件，并通过求解连续性方程获取各血液质点垂直于声束方向的速度分量（即切向速度分量）；最后，合成心脏流场速度矢量图，并实现心脏流场流线图的可视化。实验结果表明，所提方法得到的心脏流场速度矢量图和流线图能准确反映左心室所对应的时相，得到的可视化结果与Aloka提供的VFM工作站的分析结果是一致的，符合左心室流体动力学特征。所提方法作为一种通用、快速的VFM方法，不需要任何供应商的技术支持和专有软件，可以进一步推进VFM在临床工作流程中的应用。

关键词: 左心室, 彩色多普勒, 室壁运动, 流体运动, 可视化

CLC Number:

TP391

Bo PENG, Yaru LUO, Shenghua XIE, Lixue YIN. Universal vector flow mapping method combined with deep learning[J]. Journal of Computer Applications, 2021, 41(11): 3368-3375.

彭博, 罗娅茹, 谢盛华, 尹立雪. 联合深度学习的通用血流向量成像方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3368-3375.

Figures/Tables 15

Fig. 1 Schematic diagram of piecewisefunction of radial velocity and distance

Fig. 2 Schematic diagram of smoothing method

Fig. 3 Schematic diagram of polar coordinate system construction

Fig. 4 Two solutions of continuity equation

Fig. 5 Training and application process of U-Net model

Fig. 6 Structure of PWC-Net model

Tab. 1 Summary information of ultrasound simulation datasets

数据集	中心频率/MHz	采样频率/MHz	原始数据大小	数据对数
4个小球仿体数据	5	40	$2 078 × 256$	416
复杂乳腺结构数据	5	40	$3 117 × 300$	380

Tab. 1 Summary information of ultrasound simulation datasets

数据集	中心频率/MHz	采样频率/MHz	原始数据大小	数据对数
4个小球仿体数据	5	40	$2 078 × 256$	416
复杂乳腺结构数据	5	40	$3 117 × 300$	380

Tab. 2 Important training parameter information of PWC-Net model

训练阶段	训练数据集	迭代次数	训练策略	损失函数
第一阶段	4个小球仿体数据	1 200 000	$S l o n g$	L2 loss+正则项
第二阶段	复杂乳腺结构数据	600 000	$S f i n e$	L1 loss+正则项

Tab. 2 Important training parameter information of PWC-Net model

训练阶段	训练数据集	迭代次数	训练策略	损失函数
第一阶段	4个小球仿体数据	1 200 000	$S l o n g$	L2 loss+正则项
第二阶段	复杂乳腺结构数据	600 000	$S f i n e$	L1 loss+正则项

Fig. 7 Flow chart of visualization method of left ventricular blood flow field

Fig. 8 Test color-Doppler images and color compensation results

Tab. 3 U-Net model evaluation results

度量标准	指标值	度量标准	指标值
DICE	0.918 9	Precision	0.909 8
VOE	0.029 1	Recall	0.935 8
RVD	0.039 0

Fig. 9 Automatic recognition results of left ventricular contour

Fig. 10 Left ventricular wall displacement estimation results obtained by PWC-Net model

Fig. 11 Velocity vector map result comparison

Fig. 12 Plane streamline map result comparison

References 21

1	尹立雪.心腔内血液流场及流体力学状态的可视化观察及量化评价［J］.中华医学超声杂志（电子版），2009，6（3）：427-431. 10.3969/j.issn.1672-6448.2009.03.002
	YIN L X. Visualised observation and quantixed evaluation on heart flow field and hydrodynamic status ［J］. Chinese Journal of Medical Ultrasound （Electronic Version）， 2009， 6（3）： 427-431. 10.3969/j.issn.1672-6448.2009.03.002
2	OHTSUKI S， TANAK M. The flow velocity distribution from the doppler information on a plane in three-dimensional flow ［J］. Journal of Visualization， 2006， 9（1）： 69-82. 10.1007/bf03181570
3	UEJIMA T， KOIKE A， SAWADA H， et al. A new echocardiographic method for identifying vortex flow in the left ventricle： numerical validation ［J］. Ultrasound in Medicine and Biology， 2010， 36（5）： 772-788. 10.1016/j.ultrasmedbio.2010.02.017
4	GARCIA D， ÁLAMO J C DEL， TANNÉ D， et al. Two-dimensional intraventricular flow mapping by digital processing conventional color-Doppler echocardiography images ［J］. IEEE Transactions on Medical Imaging， 2010， 29（10）： 1701-1713. 10.1109/tmi.2010.2049656
5	ITATANI K， OKADA T， UEJIMA T， et al. Intraventricular flow velocity vector visualization based on the continuity equation and measurements of vorticity and wall shear stress ［J］. Japanese Journal of Applied Physics， 2013， 52（7）： 1044-1055. 10.7567/jjap.52.07hf16
6	谢盛华，尹立雪，甘建红，等.彩色多普勒图像心脏流场流线可视化方法［J］.计算机辅助设计与图形学学报，2014，26（2）：287-292. 10.3969/j.issn.1003-9775.2014.02.015
	XIE S H， YIN L X， GAN J H， et al. Visualization method of plane streamlines in cardiac flow field based on color Doppler image information ［J］. Journal of Computer-Aided Design and Computer Graphics， 2014， 26（2）： 287-292. 10.3969/j.issn.1003-9775.2014.02.015
7	ASAMI R， TANAKA T， KAWABATA K I， et al. Accuracy and limitations of vector flow mapping： left ventricular phantom validation using stereo particle image velocimetory ［J］. Journal of Echocardiography， 2017， 15（2）： 57-66. 10.1007/s12574-016-0321-5
8	TANAKA T， ASAMI R， KAWABATA K I， et al. A posteriori accuracy estimation of ultrasonic Vector-Flow Mapping （VFM）［J］. Journal of Visualization， 2017， 20（3）： 607-623. 10.1007/s12650-016-0413-3
9	ZHUANG Z M， LIU G B， DING W L， et al. Cardiac VFM visualization and analysis based on YOLO deep learning model and modified 2D continuity equation ［J］. Computerized Medical Imaging and Graphics， 2020， 82： Article No.101732. 10.1016/j.compmedimag.2020.101732
10	SUN D， YANG X， LIU M Y， et al. PWC-Net： CNNs for optical flow using pyramid， warping， and cost volume. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IIEEE， 2018： 8934-8943. 10.1109/cvpr.2018.00931
11	尹立雪.现代超声心脏电生理学［M］.北京：人民军医出版社，2007：167-178.
	YIN L X. Modern Ultrasonic in Cardiac Electrophysiology ［M］. Beijing： People’s Military Medical Press， 2007： 167-178.
12	KOBAYSHI I， MORI S， ARAKAWA M， et al. Spiral complex movements of the heart wall at the beginning of myocardial contraction detected by high frame speckle tracking ［C］// Proceedings of the 2018 IEEE International Ultrasonics Symposium. Piscataway： IEEE， 2018： 1-4. 10.1109/ultsym.2018.8579988
13	CALVO M， BONNET J L， LE ROLLE V， et al. Evaluation of three-dimensional accelerometers for the study of left ventricular contractility ［C］// Proceedings of the 2018 Computing in Cardiology Conference. Piscataway： IEEE， 2018： 1-4. 10.22489/cinc.2018.176
14	ZENG Q Y， WANG L， YANG Y L， et al. Comparison of multidirectional myocardial deformation result from pressure and volume overload in compensatory stage ［J］. IEEE Access， 2018， 6： 76429-76436. 10.1109/access.2018.2882438
15	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation ［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS9351. Berlin： Springer， 2015： 234-241.
16	LECLERC S， SMISTAD E， PEDROSA J， et al. Deep learning for segmentation using an open large-scale dataset in 2D echocardiography ［J］. IEEE Transactions on Medical Imaging， 2019， 38（9）： 2198-2210. 10.1109/tmi.2019.2900516
17	PENG B， XIAN Y H， ZHANG Q， et al. Neural-network-based motion tracking for breast ultrasound strain elastography： an initial assessment of performance and feasibility ［J］. Ultrasonic Imaging， 2020， 42（2）： 74-91. 10.1177/0161734620902527
18	DOSOVITSKIY A， FISCHER P， ILG E， et al. FlowNet： learning optical flow with convolutional networks ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 2758-2766. 10.1109/iccv.2015.316
19	GROVER P. 5-regression-loss-functions-all-machine-learners-should-know： choosing the right loss function for fitting a model ［EB/OL］. ［2021-03-08］. .
20	施仲伟.超声心动图在心血管疾病诊断中的应用［J］.临床荟萃，2009，24（19）：1671-1674.
	SHI Z W. Application of echocardiography in diagnosis of cardiovascular diseases ［J］. Clinical Focus， 2009， 24（19）： 1671-1674.
21	LEDESMA-CARBAYO M J， KYBIC J， DESCO M， et al. Spatio-temporal nonrigid registration for ultrasound cardiac motion estimation ［J］. IEEE Transactions on Medical Imaging， 2005， 24（9）： 1113-1126. 10.1109/tmi.2005.852050

[1]	LIU Li, HU Haibo, YANG Tao. Visual analysis system for exploring spatio-temporal exhibition data [J]. Journal of Computer Applications, 2020, 40(9): 2719-2727.
[2]	ZHANG Daowei, DUAN Haixin. Website fingerprinting technique based on image texture [J]. Journal of Computer Applications, 2020, 40(6): 1685-1691.
[3]	QIAO Jiewen, CHEN Wei. Adaptive color mapping and its application in void evolution visualization [J]. Journal of Computer Applications, 2020, 40(6): 1783-1792.
[4]	WANG Bo, CAI Honghao, SU Yang. Classification of malicious code variants based on VGGNet [J]. Journal of Computer Applications, 2020, 40(1): 162-167.
[5]	HE Huaiqing, ZHENG Liyuan, LIU Haohan, ZHANG Yumin. Visual analysis method for pilot eye movement data based on user-defined interest area [J]. Journal of Computer Applications, 2019, 39(9): 2683-2688.
[6]	XIE Jing, ZOU Bin, LI Shenxin, ZHAO Xiuge, QIU Yonghong. Visual decision support platform for air pollution exposure risk prevention and control [J]. Journal of Computer Applications, 2019, 39(11): 3391-3397.
[7]	JI Lianen, ZHANG Xiaolin, LIANG Shiyi, WANG Bin. Optimizing multi-slice real-time interactive visualization for out-of-core seismic volume [J]. Journal of Computer Applications, 2017, 37(9): 2621-2625.
[8]	YANG Huanhuan, LI Tianrui, CHEN Xindi. Visualization of time series data based on spiral graph [J]. Journal of Computer Applications, 2017, 37(9): 2443-2448.
[9]	CUI Di, GUO Xiaoyan, CHEN Wei. Challenges and recent progress in big data visualization [J]. Journal of Computer Applications, 2017, 37(7): 2044-2049.
[10]	TAO Pan, FU Zhongliang, ZHU Kai, WANG Lili. Echocardiogram view recognition using deep convolutional neural network [J]. Journal of Computer Applications, 2017, 37(5): 1434-1438.
[11]	XU Jie, FAN Yushun, BAI Bing. Knowledge mining and visualizing for scenic spots with probabilistic topic model [J]. Journal of Computer Applications, 2016, 36(8): 2103-2108.
[12]	ZHANG Wenying, HE Kunjin, ZHANG Rongli, LIU Yuxing. 3-D visualization and information management system design based on open scene graph [J]. Journal of Computer Applications, 2016, 36(7): 2056-2060.
[13]	WANG Xiangang, YAO Zhonghua, SONG Hanchen. MSNV:network structure visualization method based on multi-level community detection [J]. Journal of Computer Applications, 2016, 36(5): 1347-1351.
[14]	JIANG Jianchun, CHEN Huiling, DENG Lu, ZHAO Jianpeng. Configuration tool design based on control-oriented multi-core real-time operating system [J]. Journal of Computer Applications, 2016, 36(3): 765-769.
[15]	LI Zhenju, LI Xuejun, XIE Jianwei, LI Yannan. Massive terrain data storage based on HBase [J]. Journal of Computer Applications, 2015, 35(7): 1849-1853.

Universal vector flow mapping method combined with deep learning

联合深度学习的通用血流向量成像方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 21

Related Articles 15

Recommended Articles

Metrics