Journal of Computer Applications ›› 2016, Vol. 36 ›› Issue (10): 2849-2853.DOI: 10.11772/j.issn.1001-9081.2016.10.2849

Previous Articles     Next Articles

Multi-channel real-time video stitching based on circular region of interest

WANG Hanguang1,2, WANG Xuguang2, WANG Haoyuan2   

  1. 1. School of Communication and Information Engineering, Shanghai University, Shanghai 200072, China;
    2. Suzhou Institute of Nano-Tech and Nano-Bionics, Chinese Academy of Sciences, Suzhou Jiangsu 215123, China
  • Received:2016-04-11 Revised:2016-06-06 Published:2016-10-10
  • Supported by:
    BackgroundThis work is partially supported by the Prospective Joint Research Project of Industry-University-Research in Jiangsu Province (BY2014064).

基于圆形感兴趣区域多路视频实时拼接

王寒光1,2, 王旭光2, 汪浩源2   

  1. 1. 上海大学 通信与信息工程学院, 上海 200072;
    2. 中国科学院 苏州纳米技术与纳米仿生研究所, 江苏 苏州 215123
  • 通讯作者: 王旭光,E-mail:xgwang2009@sinano.ac.cn
  • 作者简介:王寒光(1990—),男,陕西渭南人,硕士研究生,主要研究方向:视频处理;王旭光(1976—),男,吉林长春人,研究员,博士,主要研究方向:图像处理、实时控制;汪浩源(1992—),女,安徽芜湖人,硕士研究生,主要研究方向:视频处理。
  • 基金资助:
    江苏省产学研前瞻性联合研究项目(BY2014064)。

Abstract: Aiming at real-time requirements and elimination ghost produced by moving object in video stitching, a method based on circular Region Of Interest (ROI) image registration was proposed by using the simplified process and Graphics Processing Unit (GPU) acceleration. Firstly, the feature extraction only occured in the ROI area, which improved the detection speed and the feature matching accuracy. Secondly, to further reduce the time cost and meet the real-time requirements for video processing, two strategies were used. On one hand, only the first frame was used for matching, while the subsequent frames used the same homography matrix to blend. On the other hand, GPU was adopted to realize hardware acceleration. Besides, when there are dynamic objects in the field of view, the graph-cut and multi-band blending algorithms were used for image blending, which can effectively eliminate ghost image. When stitching two videos of 640×480, the processing speed of the proposed method was up to 27.8 frames per second. Compared with the Speeded Up Robust Features (SURF) and Oriented features from Accelerated Segment Test (FAST) and Rotated BRIEF (ORB), the efficiency of the proposed method was increased by 26.27 times and 11.57 times respectively. Experimental results show the proposed method can be used to stitch multi-channel videos into a high quality video.

Key words: video stitching, Speeded Up Robust Feature (SURF), circular Region Of Interest (ROI), real-time, eliminating ghost

摘要: 针对视频拼接过程中面临的许多挑战,如实时性、有动态物体产生鬼影现象等,提出了一种基于圆形感兴趣区域(ROI)图像配准结合简化处理及图形处理器(GPU)加速的方法。首先,仅在ROI内提取特征点,提高了特征检测效率和匹配准确率。其次,为进一步降低时间开销,满足视频处理实时性需求,采用了两种策略:一方面,通过简化处理仅对首帧作图像配准,后续帧利用得到的单应性矩阵进行图像融合;另一方面,利用GPU多核实现并行化硬件加速。此外,当视场中有动态物体时,采用图形分割和多频带图像融合算法,有效地消除了鬼影。实验对两路640×480的视频进行拼接,该方法的处理速度可达27.8帧/秒。相对于基于加速鲁棒特征(SURF)算法的视频拼接方法,效率提高了26.27倍;相对于基于带方向的加速分段测试特征提取结合旋转的二进制鲁棒独立元素特征描述(ORB)算法的视频拼接方法,效率提高了11.57倍。实验结果表明,该方法可将多路视频实时地拼接为高质量的大场景视频。

关键词: 视频拼接, 加速鲁棒特征, 圆形感兴趣区域, 实时性, 消除鬼影

CLC Number: