《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (8): 2564-2570.DOI: 10.11772/j.issn.1001-9081.2021061061

• 多媒体计算与计算机仿真 • 上一篇    下一篇

融入时空显著性的高精度视频稳像算法

尹丽华(), 康亮, 朱文华   

  1. 上海第二工业大学 工程训练中心,上海 201209
  • 收稿日期:2021-06-24 修回日期:2022-01-27 接受日期:2022-02-09 发布日期:2022-03-16 出版日期:2022-08-10
  • 通讯作者: 尹丽华
  • 作者简介:尹丽华(1988—),女,山东省泰安人,讲师,博士,主要研究方向:计算机视觉、模式识别;
    康亮(1980—),男,黑龙江绥滨人,副教授,博士,主要研究方向:数字图像处理、机器学习;
    朱文华(1968—),男,江苏无锡人,教授,博士,主要研究方向:虚拟仿真、优化算法。
  • 基金资助:
    教育部科技发展中心产学研创新基金资助项目(2018C01059);上海第二工业大学国家自然科学基金预研项目(EGD21QD16)

High-accuracy video image stabilization algorithm incorporating temporal and spatial saliency

Lihua YIN(), Liang KANG, Wenhua ZHU   

  1. Engineering Training Center,Shanghai Polytechnic University,Shanghai 201209,China
  • Received:2021-06-24 Revised:2022-01-27 Accepted:2022-02-09 Online:2022-03-16 Published:2022-08-10
  • Contact: Lihua YIN
  • About author:YIN Lihua, born in 1988, Ph. D., lecturer. Her research interests include computer vision, pattern recognition.
    KANG Liang, born in 1980, Ph. D. associate professor. His research interests include digital image processing, machine learning.
    ZHU Wenhua, born in 1968, Ph. D. professor. His research interests include virtual simulation, optimization algorithm.
  • Supported by:
    Industry-University-Research Innovation Foundation of Science and Technology Development Center of Ministry of Education(2018C01059);National Natural Pre-Research Project of Shanghai Polytechnic University(EGD21QD16)

摘要:

为剔除复杂运动前景对视频稳像精度的干扰,同时结合时空显著性在运动目标检测上的独特优势,提出一种融入时空显著性的高精度视频稳像算法。该算法一方面通过时空显著性检测技术识别出运动目标并对其进行剔除;另一方面,采用多网格的运动路径进行运动补偿。具体包括:SURF特征点提取和匹配、时空显著性目标检测、网格划分与运动矢量计算、运动轨迹生成、多路径平滑、运动补偿等环节。实验结果表明,相较于传统的稳像算法,所提算法在稳定度(Stability)指标方面表现突出。对于有大范围运动前景干扰的视频,所提算法比RTVSM(Robust Traffic Video Stabilization Method assisted by foreground feature trajectories)的Stability指标提高了约9.6%;对于有多运动前景干扰的视频,所提算法比Bundled-paths算法的Stability指标提高了约5.8%,充分说明了所提算法对于复杂场景的稳像优势。

关键词: 视频稳像, 时空显著性, 高精度视频, 运动前景, 网格

Abstract:

In order to eliminate the interference of complicated moving foreground on the accuracy of video stabilization, and to combine with the unique advantages of temporal and spatial saliency in moving target detection, a high-accuracy video stabilization algorithm incorporating temporal and spatial saliency was proposed. In the proposed algorithm, the spatio-temporal saliency detection technology was used to identify and eliminate the moving targets. At the same time, the multi-grid motion paths were adopted for motion compensation. The proposed algorithm specifically includes: Speeded Up Robust Features (SURF) feature point extraction and matching, spatio-temporal saliency target detection, grid division and motion vector calculation, motion trajectory generation, multi-path smoothing, motion compensation and so on. Experimental results show that compared with traditional image stabilization algorithms, the proposed algorithm has outstanding performance in stability index. For videos with moving large range of foreground interference,the stability of the proposed algorithm is improved by about 9.6% compared with Robust Traffic Video Stabilization Method assisted by foreground feature trajectories (RTVSM). For videos with multiple moving foreground interference,the stability of the proposed algorithm is improved by about 5.8% compared with Bundled-paths algorithm, which fully verify the proposed algorithm’s image stability advantage.

Key words: video image stabilization, temporal and spatial saliency, high-accuracy video, moving foreground, grid

中图分类号: