Two-channel progressive feature filtering network for tampered image detection and localization

doi:10.11772/j.issn.1001-9081.2023040493

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1303-1309.DOI: 10.11772/j.issn.1001-9081.2023040493

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Two-channel progressive feature filtering network for tampered image detection and localization

Shunwang FU¹, Qian CHEN¹(), Zhi LI², Guomei WANG², Yu LU³

^1.College of Big Data and Information Engineering，Guizhou University，Guiyang Guizhou 550025，China
^2.College of Computer Science and Technology，Guizhou University，Guiyang Guizhou 550025，China
^3.Guizhou Power Grid Company Limited，Guiyang Guizhou 550002，China

Received:2023-04-28 Revised:2023-07-26 Accepted:2023-07-31 Online:2023-12-04 Published:2024-04-10
Contact: Qian CHEN
About author:FU Shunwang， born in 1996， M.S. candidate. His research interests include deep learning， image tamper detection.
CHEN Qian， born in 1981， Ph. D.， professor. Her research interests include semiconductor material， machine learning.
LI Zhi， born in 1977， Ph. D.， professor. Her research interests include information hiding， artificial intelligence， medical image analysis.
WANG Guomei， born in 1975， M. S.， associate professor. Her research interests include watermark algorithm， artificial intelligence， medical image analysis.
LU Yu， born in 1976， M. S. Her research interests include artificial intelligence， abnormal detection of power grid images.
Supported by:
National Natural Science Foundation of China(62062023)

用于篡改图像检测和定位的双通道渐进式特征过滤网络

付顺旺¹, 陈茜¹(), 李智², 王国美², 卢妤³

^1.贵州大学大数据与信息工程学院，贵阳 550025
^2.贵州大学计算机科学与技术学院，贵阳 550025
^3.贵州电网有限责任公司，贵阳 550002

通讯作者: 陈茜
作者简介:付顺旺（1996—），男，贵州遵义人，硕士研究生，CCF会员，主要研究方向：深度学习、图像篡改检测
陈茜（1981—），女，贵州贵阳人，教授，博士，主要研究方向：半导体材料、机器学习 chenzhangqianer@163.com
李智（1977—），女，贵州贵阳人，教授，博士，CCF会员，主要研究方向：信息隐藏、人工智能、医学图像分析
王国美（1975—），女，贵州贵阳人，副教授，硕士，主要研究方向：水印算法、人工智能、医学图像分析
卢妤（1976—），女，贵州贵阳人，硕士，主要研究方向：人工智能、电网图像异常检测。
基金资助:
国家自然科学基金资助项目(62062023)

Abstract

Abstract:

The existing image tamper detection networks based on deep learning often have problems such as low detection accuracy and weak algorithm transferability. To address the above issues， a two-channel progressive feature filtering network was proposed. Two channels were used to extract the two-domain features of the image in parallel， one of which was used to extract the shallow and deep features of the image spatial domain， and the other channel was used to extract the feature distribution of the image noise domain. At the same time， a progressive subtle feature screening mechanism was used to filter redundant features and gradually locate the tampered regions； in order to extract the tamper mask more accurately， a two-channel subtle feature extraction module was proposed， which combined the subtle features of the spatial domain and the noise domain to generate a more accurate tamper mask. During the decoding process， the localization ability of the network to tampered regions was improved by fusing filtered features of different scales and the contextual information of the network. The experimental results show that in terms of detection and localization， compared with the existing advanced tamper detection networks ObjectFormer， Multi-View multi-Scale Supervision Network （MVSS-Net） and Progressive Spatio-Channel Correlation Network （PSCC-Net）， the F1 score of the proposed network is increased by an 10.4， 5.9 and 12.9 percentage points on CASIA V2.0 dataset； when faced with Gaussian low-pass filtering， Gaussian noise， and JPEG compression attacks， compared with Manipulation Tracing Network （ManTra-Net） and Spatial Pyramid Attention Network （SPAN）， the Area Under Curve （AUC） of the proposed network is increased by 10.0 and 5.4 percentage points at least. It is verified that the proposed network can effectively solve the problems of low detection accuracy and poor transferability in the tamper detection algorithm.

Key words: tampered image detection, multiscale fusion, global correlation, passive forensics, residual network

摘要：

针对现有基于深度学习的篡改图像检测网络通常存在检测精度不高、算法可迁移性弱等问题，提出一种双通道渐进式特征过滤网络。利用两个通道并行提取图像的双域特征，一个通道提取图像空间域的浅层和深层特征，另一个通道提取图像噪声域的特征分布；同时，使用渐进式细微特征筛选机制过滤冗余特征，逐步定位篡改区域；为了更准确地提取篡改掩码，提出一个双输入细微特征提取模块，结合空间域和噪声域的细微特征，生成更准确的篡改掩码；在解码过程中，通过融合不同尺度的过滤特征和网络的上下文信息，提高网络对篡改区域的定位能力。实验结果表明，在检测和定位方面，与现有先进的篡改检测网络ObjectFormer、MVSS-Net（Multi-View multi-Scale Supervision Network）和PSCC-Net（Progressive Spatio-Channel Correlation Network）相比，所提网络的F1分数在CASIA V2.0数据集上分别提高10.4、5.9和12.9个百分点；面对高斯低通滤波、高斯噪声和JPEG压缩攻击时，相较于ManTra-Net（Manipulation Tracing Network）、SPAN（Spatial Pyramid Attention Network），所提网络的曲线下面积（AUC）分别至少提高了10.0、5.4个百分点。验证了所提网络可以有效解决篡改检测算法存在的检测精度不高、迁移性差等问题。

关键词: 篡改图像检测, 多尺度融合, 全局相关性, 被动取证, 残差网络

CLC Number:

TP309.2

Shunwang FU, Qian CHEN, Zhi LI, Guomei WANG, Yu LU. Two-channel progressive feature filtering network for tampered image detection and localization[J]. Journal of Computer Applications, 2024, 44(4): 1303-1309.

付顺旺, 陈茜, 李智, 王国美, 卢妤. 用于篡改图像检测和定位的双通道渐进式特征过滤网络[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1303-1309.

Figures/Tables 11

References 37

1	LAHIRI A， JAIN A K， AGRAWAL S， et al. Prior guided GAN based semantic inpainting［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 13696-13705. 10.1109/cvpr42600.2020.01371
2	LEE C-H， LIU Z， WU L， et al. MaskGAN： towards diverse and interactive facial image manipulation［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 5549-5558. 10.1109/cvpr42600.2020.00559
3	SHEN Y， GU J， TANG X， et al. Interpreting the latent space of gans for semantic face editing［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 9243-9252. 10.1109/cvpr42600.2020.00926
4	AMERINI I， URICCHIO T， BALLAN L， et al. Localization of JPEG double compression through multi-domain convolutional neural networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 53-59. 10.1109/cvprw.2017.233
5	HAN J G， PARK T H， MOON Y H， et al. Efficient Markov feature extraction method for image splicing detection using maximization and threshold expansion［J］. Journal of Electronic Imaging， 2016， 25（2）： 023031. 10.1117/1.jei.25.2.023031
6	SALLOUM R， REN Y， C-C J KUO. Image splicing localization using a Multi-Task Fully Convolutional Network （MFCN）［J］. Journal of Visual Communication and Image Representation， 2018， 51： 201-209. 10.1016/j.jvcir.2018.01.010
7	BAYAR B， STAMM M C. Constrained convolutional neural networks： a new approach towards general purpose image manipulation detection［J］. IEEE Transactions on Information Forensics and Security， 2018， 13（11）： 2691-2706. 10.1109/tifs.2018.2825953
8	COZZOLINO D， VERDOLIVA L. Noiseprint： a CNN-based camera model fingerprint［J］. IEEE Transactions on Information Forensics and Security， 2019， 15： 144-159. 10.1109/tifs.2019.2916364
9	FAN Y， CARRÉ P， FERNANDEZ-MALOIGNE C. Image splicing detection with local illumination estimation［C］// Proceedings of the 2015 IEEE International Conference on Image Processing. Piscataway： IEEE， 2015： 2940-2944. 10.1109/icip.2015.7351341
10	顾嘉城，龙英文，吉明明.基于集成生成对抗网络的视频异常事件检测方法［J］.液晶与显示，2022，37（12）：1607-1613. 10.37188/cjlcd.2022-0151
	GU J C， LONG Y W， JI M M. Video anomaly detection based on ensemble generative adversarial networks ［J］. Chinese Journal of Liquid Crystals and Displays，2022，37（12）：1607-1613. 10.37188/cjlcd.2022-0151
11	WU Y， ABD-ALMAGEED W， NATARAJAN P. BusterNet： detecting copy-move image forgery with source/target localization［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 170-184. 10.1007/978-3-030-01231-1_11
12	HUH M， LIU A， OWENS A， et al. Fighting fake news： image splice detection via learned self-consistency［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer， 2018： 101-117. 10.1007/978-3-030-01252-6_7
13	BONDI L， LAMERI S， GÜERA D， et al. Tampering detection and localization through clustering of camera-based CNN features［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1855-1864. 10.1109/cvprw.2017.232
14	BONDI L， BAROFFIO L， GÜERA D， et al. First steps toward camera model identification with convolutional neural networks［J］. IEEE Signal Processing Letters， 2017， 24（3）： 259-263. 10.1109/lsp.2016.2641006
15	SHI Z， SHEN X， KANG H， et al. Image manipulation detection and localization based on the dual-domain convolutional neural networks［J］. IEEE Access， 2018， 6： 76437-76453. 10.1109/access.2018.2883588
16	王宏，钱清，王欢，等.LKA-EfficientNet：大数据背景下融合大核注意力卷积的轻量化图像篡改定位算法［J］. 计算机应用， 2023， 43（9）：2692-2699.
	WANG H， QIAN Q， WANG H， et al. LKA-EfficientNet： lightweight image tamper location algorithm for big data based on large-core attention convolution［J］. Journal of Computer Applications， 2023， 43（9）：2692-2699.
17	T-T NG， HSU J， CHANG S-F. Columbia image splicing detection evaluation dataset［DB/OL］. ［2023-04-01］. .
18	WEN B， ZHU Y， SUBRAMANIAN R， et al. COVERAGE — a novel database for copy-move forgery detection［C］// Proceedings of the 2016 IEEE International Conference on Image Processing. Piscataway： IEEE， 2016： 161-165. 10.1109/icip.2016.7532339
19	DONG J， WANG W， TAN T. CASIA image tampering detection evaluation database［C］// Proceedings of the 2013 IEEE China Summit and International Conference on Signal and Information Processing. Piscataway： IEEE， 2013： 422-426. 10.1109/chinasip.2013.6625374
20	GUAN H， KOZAK M， ROBERTSON E， et al. MFC datasets： large-scale benchmark datasets for media forensic challenge evaluation［C］// Proceedings of the 2019 IEEE Winter Applications of Computer Vision Workshops. Piscataway： IEEE， 2019： 63-72. 10.1109/wacvw.2019.00018
21	WANG J， WU Z， CHEN J， et al. ObjectFormer for image manipulation detection and localization［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 2364-2373. 10.1109/cvpr52688.2022.00240
22	LIU X， LIU Y， CHEN J， et al. PSCC-Net： progressive spatio-channel correlation network for image manipulation detection and localization［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2022， 32（11）： 7505-7517. 10.1109/tcsvt.2022.3189545
23	CHEN X， DONG C， JI J， et al. Image manipulation detection by multi-view multi-scale supervision［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 14185-14193. 10.1109/iccv48922.2021.01392
24	WU Y， ABDALMAGEED W， NATARAJAN P. ManTra-Net： manipulation tracing network for detection and localization of image forgeries with anomalous features［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9543-9552. 10.1109/cvpr.2019.00977
25	HU X， ZHANG Z， JIANG Z， et al. SPAN： spatial pyramid attention network for image manipulation localization［C］// Proceedings of the 16th European Conference on Computer Vision. Cham： Springer， 2020： 312-328. 10.1007/978-3-030-58589-1_19
26	CHIERCHIA G， POGGI G， SANSONE C， et al. PRNU-based forgery detection with regularity constraints and global optimization［C］// Proceedings of the 2013 IEEE 15th International Workshop on Multimedia Signal Processing. Piscataway： IEEE， 2013： 236-241. 10.1109/mmsp.2013.6659294
27	CHIERCHIA G， POGGI G， SANSONE C， et al. A Bayesian-MRF approach for PRNU-based image forgery detection［J］. IEEE Transactions on Information Forensics and Security， 2014， 9（4）： 554-567. 10.1109/tifs.2014.2302078
28	ZHOU P， HAN X， MORARIU V I， et al. Learning rich features for image manipulation detection［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 1053-1061. 10.1109/cvpr.2018.00116
29	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
30	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
31	WANG X， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7794-7803. 10.1109/cvpr.2018.00813
32	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
33	YANG M， YU K， ZHANG C， et al. DenseASPP for semantic segmentation in street scenes［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 3684-3692. 10.1109/cvpr.2018.00388
34	ZHANG Y， LI K， LI K， et al. Image super-resolution using very deep residual channel attention networks［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer，2018： 294-310. 10.1007/978-3-030-01234-2_18
35	WARIF N B A， IDRIS M Y I， WAHAB A W A， et al. An evaluation of error level analysis in image forensics［C］// Proceedings of the 2015 5th IEEE International Conference on System Engineering and Technology. Piscataway： IEEE， 2015： 23-28. 10.1109/icsengt.2015.7412439
36	MAHDIAN B， SAIC S. Using noise inconsistencies for blind image forensics［J］. Image and Vision Computing， 2009， 27（10）： 1497-1503. 10.1016/j.imavis.2009.02.001
37	FERRARA P， BIANCHI T， DE ROSA A， et al. Image forgery localization via fine-grained analysis of CFA artifacts［J］. IEEE Transactions on Information Forensics and Security， 2012， 7（5）： 1566-1577. 10.1109/tifs.2012.2202227

数据集	图像数	复制-粘贴数	图像拼接数	图像删除数
Columbia^［17］	180	0	180	0
Coverage^［18］	100	100	0	0
CASIA V2.0^［19］	5 063	3 235	1 828	0
NIST16^［20］	564	68	288	208

数据集	图像数	复制-粘贴数	图像拼接数	图像删除数
Columbia^［17］	180	0	180	0
Coverage^［18］	100	100	0	0
CASIA V2.0^［19］	5 063	3 235	1 828	0
NIST16^［20］	564	68	288	208

网络	CASIA V2.0		Coverage		NIST16		Columbia
网络	AUC	F1	AUC	F1	AUC	F1	AUC	F1
ELA^［35］	61.3	21.4	58.3	22.2	42.9	23.6	—	—
NOI1^［36］	61.2	26.3	58.7	26.9	48.7	28.5	58.6	—
CFA1^［37］	52.2	20.7	48.5	19.0	50.1	17.4	48.7	—
ManTra-Net^［24］	81.7	48.1	79.5	—	84.5	82.0	82.4	—
MVSS-Net^［23］	86.6	62.4	73.1	22.4	83.9	75.3	98.0	80.2
SPAN^［25］	83.8	38.2	93.7	55.8	83.6	29.0	—	—
PSCC-Net^［22］	87.5	55.4	94.1	72.3	99.6	81.9	98.2	98.1
ObjectFormer^［21］	88.2	57.9	95.7	75.8	99.6	82.4	95.5	—
本文网络	89.1	68.3	81.5	65.1	93.8	90.2	99.3	99.1

网络	CASIA V2.0		Coverage		NIST16		Columbia
网络	AUC	F1	AUC	F1	AUC	F1	AUC	F1
ELA^［35］	61.3	21.4	58.3	22.2	42.9	23.6	—	—
NOI1^［36］	61.2	26.3	58.7	26.9	48.7	28.5	58.6	—
CFA1^［37］	52.2	20.7	48.5	19.0	50.1	17.4	48.7	—
ManTra-Net^［24］	81.7	48.1	79.5	—	84.5	82.0	82.4	—
MVSS-Net^［23］	86.6	62.4	73.1	22.4	83.9	75.3	98.0	80.2
SPAN^［25］	83.8	38.2	93.7	55.8	83.6	29.0	—	—
PSCC-Net^［22］	87.5	55.4	94.1	72.3	99.6	81.9	98.2	98.1
ObjectFormer^［21］	88.2	57.9	95.7	75.8	99.6	82.4	95.5	—
本文网络	89.1	68.3	81.5	65.1	93.8	90.2	99.3	99.1

网络结构	CASIA V2.0		Coverage		NIST16		Columbia
网络结构	AUC	F1	AUC	F1	AUC	F1	AUC	F1
M₅	77.0	59.2	75.6	53.9	89.8	83.1	94.1		87.2
M₁+M₅	78.5	61.3	76.2	54.6	92.4	84.7	97.7		93.8
M₁+M₂+M₅	78.7	62.1	78.6	59.2	92.6	85.6	98.3		96.0
M₁+M₂+M₃+M₅	80.2	64.6	80.3	63.8	92.8	87.5	98.9		97.5
M₁+M₂+M₃+M₄+M₅	89.1	68.3	81.5	65.1	93.8	90.2	99.3		99.1

Two-channel progressive feature filtering network for tampered image detection and localization

用于篡改图像检测和定位的双通道渐进式特征过滤网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 37

Related Articles 15

Recommended Articles

Metrics

网络设置	CASIA V2.0		Coverage		NIST16		Columbia
网络设置	AUC	F1	AUC	F1	AUC	F1	AUC	F1
单ResNet-50+3×3卷积	75.3	56.7	74.3	47.9	86.7	76.1	98.1	94.1
双ResNet-50+3×3卷积	78.6	62.8	76.1	53.4	87.3	78.9	98.6	94.2
单ResNet-50+双输入细微特征模块	79.4	63.9	80.1	59.5	93.6	89.1	99.3	96.0
双ResNet-50+双输入细微特征模块	89.1	68.3	81.5	65.1	93.8	90.2	99.3	99.1

攻击类型	参数	AUC值/%
攻击类型	参数	ManTra-Net	SPAN	本文网络
无操作	—	84.5	83.6	93.8
高斯滤波	k=3	77.4	83.1	91.5
高斯滤波	k=15	74.5	79.1	84.5
高斯噪声	σ=3	67.4	75.1	92.1
高斯噪声	σ=15	58.5	67.2	85.7
JPEG压缩	QF= 100	77.9	83.5	93.6
JPEG压缩	QF= 50	74.3	80.6	90.1

[1]	Yuanjiong LIU, Maozheng HE, Yibin HUANG, Cheng QIAN. Ship identification model based on ResNet50 and improved attention mechanism [J]. Journal of Computer Applications, 2024, 44(6): 1935-1941.
[2]	Lin GUO, Kunhu LIU, Chenyang MA, Youxue LAI, Yingfen XU. Image super-resolution reconstruction based on residual attention network with receptive field expansion [J]. Journal of Computer Applications, 2024, 44(5): 1579-1587.
[3]	Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316.
[4]	Jingxian ZHOU, Xina LI. UAV detection and recognition based on improved convolutional neural network and radio frequency fingerprint [J]. Journal of Computer Applications, 2024, 44(3): 876-882.
[5]	Hong WANG, Qing QIAN, Huan WANG, Yong LONG. Lightweight image tamper localization algorithm based on large kernel attention convolution [J]. Journal of Computer Applications, 2023, 43(9): 2692-2699.
[6]	Xueyu HUANG, Huaiyu HE, Huimin LIN, Jinshui CHEN. Classification and recognition method of copper alloy metallograph based on feature aggregation [J]. Journal of Computer Applications, 2023, 43(8): 2593-2601.
[7]	Huibin ZHANG, Liping FENG, Yaojun HAO, Yining WANG. Ancient mural dynasty identification based on attention mechanism and transfer learning [J]. Journal of Computer Applications, 2023, 43(6): 1826-1832.
[8]	Lihua SHEN, Bo LI. Super-resolution reconstruction of lung CT images based on feature pyramid network and dense network [J]. Journal of Computer Applications, 2023, 43(5): 1612-1619.
[9]	Qihong SONG, Jianxun LIU, Haize HU, Xiangping ZHANG. Code search model based on collaborative fusion network [J]. Journal of Computer Applications, 2023, 43(12): 3896-3902.
[10]	Zhiang ZHANG, Guangzhong LIAO. Multi-scale feature enhanced retinal vessel segmentation algorithm based on U-Net [J]. Journal of Computer Applications, 2023, 43(10): 3275-3281.
[11]	Liefa LIAO, Zhiming LI, Saisai ZHANG. Image retrieval method based on deep residual network and iterative quantization hashing [J]. Journal of Computer Applications, 2022, 42(9): 2845-2852.
[12]	Huaiqing HE, Jianqing YAN, Kanghua HUI. Lightweight face recognition method based on deep residual network [J]. Journal of Computer Applications, 2022, 42(7): 2030-2036.
[13]	Yang ZHANG, Jiangbo HAO. Malicious code detection method based on attention mechanism and residual network [J]. Journal of Computer Applications, 2022, 42(6): 1708-1715.
[14]	Mingyu DONG, Diqun YAN. Detection algorithm of audio scene sound replacement falsification based on ResNet [J]. Journal of Computer Applications, 2022, 42(6): 1724-1728.
[15]	Huifeng WANG, Yan XU, Yiming WEI, Huizhen WANG. Image super-resolution reconstruction based on parallel convolution and residual network [J]. Journal of Computer Applications, 2022, 42(5): 1570-1576.

网络	帧率
ManTra-Net	2.8
MVSS-Net	20.1
本文网络	12.9

网络	帧率
ManTra-Net	2.8
MVSS-Net	20.1
本文网络	12.9