计算机应用 ›› 2019, Vol. 39 ›› Issue (5): 1473-1479.DOI: 10.11772/j.issn.1001-9081.2018081801

• 虚拟现实与多媒体计算 • 上一篇    下一篇

基于自适应可分离卷积核的视频压缩伪影去除算法

聂可卉1,2, 刘文哲3, 童同3, 杜民2, 高钦泉1,2,3   

  1. 1. 福州大学 物理与信息工程学院, 福州 350116;
    2. 福建省医疗器械与医药技术重点实验室(福州大学), 福州 350116;
    3. 福建帝视信息科技有限公司, 福州 350001
  • 收稿日期:2018-09-04 修回日期:2018-12-19 出版日期:2019-05-10 发布日期:2019-05-14
  • 通讯作者: 高钦泉
  • 作者简介:聂可卉(1994-),女,江西吉安人,硕士研究生,主要研究方向:计算机视觉、图像处理;刘文哲(1993-),男,福建三明人,硕士,主要研究方向:计算机视觉、图像处理;童同(1986-),男,安徽安庆人,博士,主要研究方向:计算机视觉、医学影像处理、脑疾病辅助诊断;杜民(1955-),女,福建泉州人,教授,博士,主要研究方向:传感技术、生物医学仪器;高钦泉(1986-),男,福建福州人,副教授,博士,主要研究方向:计算机视觉、增强现实、医学影像处理。

Video compression artifact removal algorithm based on adaptive separable convolution network

NIE Kehui1,2, LIU Wenzhe3, TONG Tong3, DU Min2, GAO Qinquan1,2,3   

  1. 1. School of Physics and Information Engineering, Fuzhou University, Fuzhou Fujian 350116, China;
    2. Key Laboratory of Medical Instrumentation & Pharmaceutical Technology of Fujian Province(Fuzhou University), Fuzhou Fujian 350116, China;
    3. Imperial Vision Technology Company, Fuzhou Fujian 350001, China
  • Received:2018-09-04 Revised:2018-12-19 Online:2019-05-10 Published:2019-05-14

摘要: 针对目前视频质量增强和超分辨率重建等任务中常采用的光流估计相关算法只能估计像素点间线性运动的问题,提出了一种新型多帧去压缩伪影网络结构。该网络由运动补偿模块和去压缩伪影模块组成。运动补偿模块采用自适应可分离卷积代替传统的光流估计算法,能够很好地处理光流法不能解决的像素点间的曲线运动问题。对于不同视频帧,运动补偿模块预测出符合该图像结构和像素局部位移的卷积核,通过局部卷积的方式实现对后一帧像素的运动偏移估计和像素补偿。将得到的运动补偿帧和原始后一帧联结起来作为去压缩伪影模块的输入,通过融合包含不同像素信息的两视频帧,得到对该帧去除压缩伪影后的结果。与目前最先进的多帧质量增强(MFQE)算法在相同的训练集和测试集上训练并测试,实验结果表明,峰值信噪比提升(ΔPSNR)较MFQE最大增加0.44 dB,平均增加0.32 dB,验证了所提出网络具有良好的去除视频压缩伪影的效果。

关键词: 视频质量增强, 光流估计, 运动补偿, 自适应可分离卷积, 去视频压缩伪影

Abstract: The existing optical flow estimation methods, which are frequently used in video quality enhancement and super-resolution reconstruction tasks, can only estimate the linear motion between pixels. In order to solve this problem, a new multi-frame compression artifact removal network architecture was proposed. The network consisted of motion compensation module and compression artifact removal module. With the traditional optical flow estimation algorithms replaced with the adaptive separable convolution, the motion compensation module was able to handle with the curvilinear motion between pixels, which was not able to be well solved by optical flow methods. For each video frame, a corresponding convolutional kernel was generated by the motion compensation module based on the image structure and the local displacement of pixels. After that, motion offsets were estimated and pixels were compensated in the next frame by means of local convolution. The obtained compensated frame and the original next frame were combined together as input for the compression artifact removal module. By fusing different pixel information of the two frames, the compression artifacts of the original frame were removed. Compared with the state-of-the-art Multi-Frame Quality Enhancement (MFQE) algorithm on the same training and testing datasets, the proposed network has the improvement of Peak Signal-to-Noise Ratio (ΔPSNR) increased by 0.44 dB at most and 0.32 dB on average. The experimental results demonstrate that the proposed network performs well in removing video compression artifacts.

Key words: video quality enhancement, optical flow estimation, motion compensation, adaptive separable convolution, video compression artifact removal

中图分类号: