Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (2): 545-551.DOI: 10.11772/j.issn.1001-9081.2021122107

• Frontier and comprehensive applications • Previous Articles    

U-shaped feature pyramid network for image inpainting forensics

Wanli SHEN, Yujin ZHANG(), Wan HU   

  1. School of Electronic and Electrical Engineering,Shanghai University of Engineering Science,Shanghai 201620,China
  • Received:2021-12-13 Revised:2022-03-08 Accepted:2022-03-15 Online:2023-02-08 Published:2023-02-10
  • Contact: Yujin ZHANG
  • About author:SHEN Wanli, born in 1997, M. S. candidate. His research interests include image processing, deep learning, image inpainting forensics.
    HU Wan, born in 1996, M. S. candidate. His research interests include image filtering forensics.
  • Supported by:
    Natural Science Foundation of Shanghai(17ZR1411900)


沈万里, 张玉金(), 胡万   

  1. 上海工程技术大学 电子电气工程学院,上海 201620
  • 通讯作者: 张玉金
  • 作者简介:沈万里(1997—),男,上海人,硕士研究生,CCF会员,主要研究方向:图像处理、深度学习、图像修复取证
  • 基金资助:


Image inpainting is a common method of image tampering. Image inpainting methods based on deep learning can generate more complex structures and even new objects, making image inpainting forensics more challenging. Therefore, an end-to-end U-shaped Feature Pyramid Network (FPN) was proposed for image inpainting forensics. Firstly, multi-scale feature extraction was performed through the from-top-to-down VGG16 module, and then the from-bottom-to-up feature pyramid architecture was used to carry out up-sampling of the fused feature maps, and a U-shaped structure was formed by the overall process. Next, the global and local attention mechanisms were combined to highlight the inpainting traces. Finally, the fusion loss function was used to improve the prediction rate of the repaired area. Experimental results show that the proposed method achieves an average F1-score and Intersection over Union (IoU) value of 0.791 9 and 0.747 2 respectively on various deep inpainting datasets. Compared with the existing Localization of Diffusion-based Inpainting (LDI), Patch-based Convolutional Neural Network (Patch-CNN) and High-Pass Fully Convolutional Network (HP-FCN) methods, the proposed method has better generalization ability, and also has stronger robustness to JPEG compression.

Key words: digital image forensics, deep image inpainting, tampering detection, Feature Pyramid Network (FPN), fusion loss


图像修复是一种常见的图像篡改手段,而基于深度学习的图像修复方法能生成更复杂的结构乃至新的对象,使得图像修复取证工作更具有挑战性。因此,提出一种端到端的面向图像修复取证的U型特征金字塔网络(FPN)。首先,通过自上而下的VGG16模块进行多尺度特征提取,并利用自下而上的特征金字塔架构对融合后的特征图进行上采样,整体流程形成U型结构;然后,结合全局和局部注意力机制凸显修复痕迹;最后,使用融合损失函数以提高修复区域的预测率。实验结果表明,所提方法在多种深度修复数据集上的平均F1分数和IoU值分别为0.791 9和0.747 2,与现有的基于扩散的数字图像修复定位(LDI)、基于图像块的深度修复取证方法(Patch-CNN)和基于高通全卷积神经网络(HP-FCN)方法相比,所提方法具有更好的泛化能力,且对JPEG压缩也具有较强的鲁棒性。

关键词: 数字图像取证, 深度图像修复, 篡改检测, 特征金字塔网络, 融合损失

CLC Number: