基于圆形感兴趣区域多路视频实时拼接

doi:10.11772/j.issn.1001-9081.2016.10.2849

计算机应用 ›› 2016, Vol. 36 ›› Issue (10): 2849-2853.DOI: 10.11772/j.issn.1001-9081.2016.10.2849

基于圆形感兴趣区域多路视频实时拼接

王寒光^1,2, 王旭光², 汪浩源²

1. 上海大学通信与信息工程学院, 上海 200072;
2. 中国科学院苏州纳米技术与纳米仿生研究所, 江苏苏州 215123

收稿日期:2016-04-11 修回日期:2016-06-06 发布日期:2016-10-10
通讯作者: 王旭光,E-mail:xgwang2009@sinano.ac.cn
作者简介:王寒光(1990—),男,陕西渭南人,硕士研究生,主要研究方向:视频处理;王旭光(1976—),男,吉林长春人,研究员,博士,主要研究方向:图像处理、实时控制;汪浩源(1992—),女,安徽芜湖人,硕士研究生,主要研究方向:视频处理。
基金资助:
江苏省产学研前瞻性联合研究项目（BY2014064）。

Multi-channel real-time video stitching based on circular region of interest

WANG Hanguang^1,2, WANG Xuguang², WANG Haoyuan²

1. School of Communication and Information Engineering, Shanghai University, Shanghai 200072, China;
2. Suzhou Institute of Nano-Tech and Nano-Bionics, Chinese Academy of Sciences, Suzhou Jiangsu 215123, China

Received:2016-04-11 Revised:2016-06-06 Published:2016-10-10
Supported by:
BackgroundThis work is partially supported by the Prospective Joint Research Project of Industry-University-Research in Jiangsu Province (BY2014064).

摘要/Abstract

摘要： 针对视频拼接过程中面临的许多挑战，如实时性、有动态物体产生鬼影现象等，提出了一种基于圆形感兴趣区域（ROI）图像配准结合简化处理及图形处理器（GPU）加速的方法。首先，仅在ROI内提取特征点，提高了特征检测效率和匹配准确率。其次，为进一步降低时间开销，满足视频处理实时性需求，采用了两种策略：一方面，通过简化处理仅对首帧作图像配准，后续帧利用得到的单应性矩阵进行图像融合；另一方面，利用GPU多核实现并行化硬件加速。此外，当视场中有动态物体时，采用图形分割和多频带图像融合算法，有效地消除了鬼影。实验对两路640×480的视频进行拼接，该方法的处理速度可达27.8帧/秒。相对于基于加速鲁棒特征（SURF）算法的视频拼接方法，效率提高了26.27倍；相对于基于带方向的加速分段测试特征提取结合旋转的二进制鲁棒独立元素特征描述（ORB）算法的视频拼接方法，效率提高了11.57倍。实验结果表明，该方法可将多路视频实时地拼接为高质量的大场景视频。

关键词: 视频拼接, 加速鲁棒特征, 圆形感兴趣区域, 实时性, 消除鬼影

Abstract: Aiming at real-time requirements and elimination ghost produced by moving object in video stitching, a method based on circular Region Of Interest (ROI) image registration was proposed by using the simplified process and Graphics Processing Unit (GPU) acceleration. Firstly, the feature extraction only occured in the ROI area, which improved the detection speed and the feature matching accuracy. Secondly, to further reduce the time cost and meet the real-time requirements for video processing, two strategies were used. On one hand, only the first frame was used for matching, while the subsequent frames used the same homography matrix to blend. On the other hand, GPU was adopted to realize hardware acceleration. Besides, when there are dynamic objects in the field of view, the graph-cut and multi-band blending algorithms were used for image blending, which can effectively eliminate ghost image. When stitching two videos of 640×480, the processing speed of the proposed method was up to 27.8 frames per second. Compared with the Speeded Up Robust Features (SURF) and Oriented features from Accelerated Segment Test (FAST) and Rotated BRIEF (ORB), the efficiency of the proposed method was increased by 26.27 times and 11.57 times respectively. Experimental results show the proposed method can be used to stitch multi-channel videos into a high quality video.

Key words: video stitching, Speeded Up Robust Feature (SURF), circular Region Of Interest (ROI), real-time, eliminating ghost

中图分类号:

TP751.1

王寒光, 王旭光, 汪浩源. 基于圆形感兴趣区域多路视频实时拼接[J]. 计算机应用, 2016, 36(10): 2849-2853.

WANG Hanguang, WANG Xuguang, WANG Haoyuan. Multi-channel real-time video stitching based on circular region of interest[J]. Journal of Computer Applications, 2016, 36(10): 2849-2853.

参考文献

[1] LI J, XU W, ZHANG J, et al. Efficient video stitching based on fast structure deformation[J]. IEEE Transactions on Cybernetics, 2015, 45(12): 2707-2719.
[2] LEE K Y, SIM J Y. Robust video stitching using adaptive pixel transfer[C]//Proceedings of the 2015 IEEE International Conference on Image Processing. Piscataway, NJ: IEEE, 2015: 813-817.
[3] JIANG W, GU J. Video stitching with spatial-temporal content-preserving warping[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ: IEEE,2015: 42-48.
[4] 首照宇, 欧阳宁, 张华俊等. 基于SURF和动态ROI的实时视频拼接[J]. 计算机工程与设计, 2013, 34(3): 998-1003.(SHOU Z Y, OUYANG N, ZHANG H J, et al. Research on real-time video stitching based on SURF and dynamic ROI[J]. Computer Engineering and Design, 2013, 34(3): 998-1003.)
[5] ZHANG Y, JIA K, LIU P. Video stitch algorithm based on dynamic foreground extraction[C]//CISP 2009: Proceedings of the 2nd International Congress on Image and Signal Processing. Piscataway, NJ: IEEE, 2009: 1-5.
[6] 雷飞, 王文学, 王雪丽等. 基于改进SURF的实时视频拼接方法[J]. 计算机技术与发展, 2015, 25(3): 32-35.(LEI F, WANG W X, WANG X L, et al. Real-time video stitching method based on improved SURF[J]. Computer Technology and Development, 2015, 25(3): 32-35.)
[7] YIN X, LI W, LIU Y, et al. FPGA-based real time video stitching method for video surveillance[J]. Optik — International Journal for Light and Electron Optics, 2015, 126(21): 2804-2808.
[8] LI J, YANG T, YU J, et al. Fast aerial video stitching[EB/OL].[2015-04-24]. http://www.intechopen.com/download/pdf/pdfs_id/47643.
[9] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[10] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359.
[11] SINHA S N, FRAHM J M, POLLEFEYS M, et al. Feature tracking and matching in video using programmable graphics hardware[J]. Machine Vision and Applications, 2011, 22(1): 207-217.
[12] HEYMANN S, MULLER K, SMOLIC A, et al. SIFT implementation and optimization for general-purpose GPU[EB/OL].[2014-11-15]. HTTP://wscg.zcu.cz/wscg/Papers_2007/Full/G03-full.pdf.
[13] ACHARYA K A, BABU R V, VADHIYAR S S. A real-time implementation of SIFT using GPU[J/OL]. Journal of Real-Time Image Processing, 2014: 1-11 [2015-05-23].http://link.springer.com/article/10.1007/s11554-014-0446-6.
[14] CHOU C H, LIU P, WU T Y, et al. Implementation of parallel computing FAST algorithm on mobile GPU[C]//Proceedings of the 2012 International Conference on Electrical and Electronics Engineering, LNEE 238. Berlin: Springer,2014: 1275-1281.
[15] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE,2011: 2564-2571.
[16] 李小红, 谢成明, 贾易臻等. 基于ORB特征的快速目标检测算法[J]. 电子测量与仪器学报, 2013, 27(5): 455-460.(LI X H, XIE C M, JIA Y Z, et al. Rapid moving object detection algorithm based on ORB features[J]. Journal of Electronic Measurement and Instrument, 2013, 27(5): 455-460.)
[17] XU J, CHANG H, YANG S, et al. Fast feature-based video stabilization without accumulative global motion estimation[J]. IEEE Transactions on Consumer Electronics, 2012, 58(3): 993-999.
[18] SARVAIYA J N, PATNAIK S, KOTHARI K. Image registration using log polar transform and phase correlation to recover higher scale[J]. Journal of Pattern Recognition Research, 2012, 7(1): 90-105.
[19] 孙步强, 静大海. 相位相关法实现视频实时拼接的应用与优化[J]. 科学技术与工程, 2012, 20(26): 6618-6621.(SUN B Q, JING D H. Application and optimization of phase correlation algorithm in real-time video mosaic[J]. Science Technology and Engineering, 2012, 20(26): 6618-6621.)
[20] 方俊伟, 翟超, 金熠. 相位相关法实现图像拼接的仿真与优化[J]. 计算机应用与软件, 2008,25(1): 207-208.(FANG J W, ZHAI C, JIN Y. Simulation and optimization of phase correlation algorithm in image mosaics[J]. Computer Applications and Software, 2008,25(1): 207-208.)
[21] SZELISKI R. Computer Vision: Algorithms and Applications[M]. Berlin: Springer, 2010:327-335.
[22] KWATRA V, SCHODL A, ESSA I, et al. Graphcut textures: image and video synthesis using graph cuts[J]. ACM Transactions on Graphics, 2003, 22(3): 277-286.
[23] BURT P J, ADELSON E H. The Laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications, 1983, 31(4): 532-540.
[24] BURT P J, ADELSON E H. A multiresolution spline with application to image mosaics[J]. ACM Transactions on Graphics, 1983, 2(4): 217-236.

基于圆形感兴趣区域多路视频实时拼接

Multi-channel real-time video stitching based on circular region of interest

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	刘晓光, 靳少康, 韦子辉, 梁铁, 王洪瑞, 刘秀玲. 基于阈值和极端随机树的实时跌倒检测方法[J]. 计算机应用, 2021, 41(9): 2761-2766.
[2]	李南云, 王旭光, 吴华强, 何青林. 基于灰度塔评分的匹配模型构建在无人机网络视频拼接中的应用[J]. 计算机应用, 2019, 39(5): 1480-1484.
[3]	赵宏, 常兆斌, 王乐. 基于词法特征的恶意域名快速检测算法[J]. 计算机应用, 2019, 39(1): 227-231.
[4]	罗元, 李慧敏, 张毅. 基于兴趣点定位的局部方向模式人脸识别方法[J]. 计算机应用, 2017, 37(8): 2248-2252.
[5]	李卓, 刘洁瑜, 李辉, 周小刚, 李维鹏. 基于ORB-LATCH的特征检测与描述算法[J]. 计算机应用, 2017, 37(6): 1759-1762.
[6]	乔通, 赵卓峰, 丁维龙. 面向套牌甄别的流式计算系统[J]. 计算机应用, 2017, 37(1): 153-158.
[7]	张靖, 李小鹏, 王衡军, 李俊全, 郁滨. 基于攻击规划图的实时报警关联方法[J]. 计算机应用, 2016, 36(6): 1538-1543.
[8]	白晓红, 温静, 赵雪, 陈金广. 基于加速鲁棒特征和多示例学习的目标跟踪算法[J]. 计算机应用, 2016, 36(11): 2974-2978.
[9]	任秀丽, 彦琨. 交通监控中基于模糊聚类的无线传感网MAC协议[J]. 计算机应用, 2016, 36(10): 2653-2658.
[10]	曹彦珏, 安博文, 李启明. 基于后处理的实时景深模拟与应用[J]. 计算机应用, 2015, 35(5): 1439-1443.
[11]	胡昭华, 邢卫国, 何军, 张秀再. 多通道核相关滤波的实时跟踪方法[J]. 计算机应用, 2015, 35(12): 3544-3549.
[12]	关国栋, 滕飞, 杨燕. 基于心跳超时机制的Hadoop实时容错技术[J]. 计算机应用, 2015, 35(10): 2784-2788.
[13]	陈燕薛原杨若瑜. 任意衣物和人体的3D试穿实时仿真[J]. 计算机应用, 2014, 34(1): 124-128.
[14]	施鹏熊岳山徐凯谭珂潘新华. 虚拟肝脏手术中实时动态渗血效果模拟[J]. 计算机应用, 2013, 33(10): 2911-2913.
[15]	孔令晶曾华燊李耀. 等级OSPF网的安全保护方案[J]. 计算机应用, 2013, 33(08): 2212-2217.