Active defense against face forgery based on attention mask and feature extraction

doi:10.11772/j.issn.1001-9081.2024030364

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (3): 904-910.DOI: 10.11772/j.issn.1001-9081.2024030364

• Cyber security • Previous Articles Next Articles

Active defense against face forgery based on attention mask and feature extraction

Yu WANG, Xianjin FANG(), Gaoming YANG, Yifeng DING, Xinlu YANG

School of Computer Science and Engineering，Anhui University of Science and Technology，Huainan Anhui 232001，China

Received:2024-04-02 Revised:2024-06-25 Accepted:2024-06-26 Online:2024-08-13 Published:2025-03-10
Contact: Xianjin FANG
About author:WANG Yu， born in 1999， M. S. candidate. Her research interests include active defense， fake face detection.
YANG Gaoming， born in 1974， Ph. D.， professor. His research interests include privacy protection， machine learning， fake face detection.
DING Yifeng， born in 1999， M. S. candidate. His research interests include computer vision， image processing， knowledge distillation.
YANG Xinlu， born in 1998， M. S. Her research interests include deepfake detection and defense.
Supported by:
National Natural Science Foundation of China(52374155);Anhui Provincial Major Science and Technology Project(18030901025);Anhui Provincial Natural Science Foundation(2308085MF218);Natural Science Research Project of Colleges and Universities in Anhui Province(2022AH040113)

基于注意力掩码与特征提取的人脸伪造主动防御

王瑜, 方贤进(), 杨高明, 丁一峰, 杨新露

安徽理工大学计算机科学与工程学院，安徽淮南 232001

通讯作者: 方贤进
作者简介:王瑜（1999—），女，安徽宿州人，硕士研究生，主要研究方向：主动防御、伪造人脸检测
杨高明（1974—），男，安徽临泉人，教授，博士生导师，博士，主要研究方向：隐私保护、机器学习、伪造人脸检测
丁一峰（1999—），男，安徽芜湖人，硕士研究生，主要研究方向：计算机视觉、图像处理、知识蒸馏
杨新露（1998—），女，山东滨州人，硕士，主要研究方向：深度伪造检测与防御。
基金资助:
国家自然科学基金资助项目(52374155);安徽省科技重大专项(18030901025);安徽省自然科学基金资助项目(2308085MF218);安徽省高等学校自然科学研究项目(2022AH040113)

Abstract

Abstract:

To address the issue of unauthorized forgery or tampering of facial images， an active defense method based on attention mask and feature extraction was proposed. This method was designed to take offensive measures to interfere with forgery models by adding adversarial examples into the image， so that the image was prevented forgery from the source and the visual quality of the protected image was enhanced. Firstly， an improved gradient descent method was employed to generate and add adversarial perturbations to the original image， resulting in the generation of a blurred false image after forgery processing the original image. At the same time， the attention mask was incorporated into the generator to enhance key feature channels， thereby reducing the influence of complex backgrounds and lighting. Additionally， the VGG16 pre-trained network was utilized to extract image features， thereby improving the visual quality of adversarial images at feature map level. Experimental results on CelebFaces Attributes （CelebA） dataset and Radboud Faces Database （RaFD） dataset show that， for StarGAN， the defense success rates of the proposed model are 99.80% and 99.63% respectively. Compared with the baseline method based on spread-spectrum adversarial attack， the proposed method has the visual quality of generated adversarial images improved by 30.86% and 26.63% respectively on Structure Similarity Index Measure （SSIM）， and the Peak Signal-to-Noise Ratio （PSNR） improved by 34.80% and 36.15% respectively. The above indicates that the proposed method defends against face image forgery effectively while enhancing the visual quality of adversarial images.

Key words: face forgery, active defense, attention mask, adversarial example, feature extraction

摘要：

为了解决人脸图像在未经授权情况下被伪造或篡改的问题，提出一种基于注意力掩码与特征提取的人脸伪造主动防御方法。该方法旨在采取攻击性措施，向图像中加入可干扰伪造模型的对抗样本，从源头上预防图像被伪造，同时提高被保护图像的视觉质量。首先，采用改进的梯度下降法生成对抗扰动并将这些扰动添加至原始图像，使原始图像在经过伪造处理后生成模糊的虚假图像；同时，在生成器中增添注意力掩码，以增强关键特征通道，从而降低复杂背景和光照带来的影响；其次，使用VGG16预训练网络提取图像特征，在特征图层面提升对抗图像的视觉质量。在名人人脸属性（CelebA）数据集和Radboud面孔数据库（RaFD）数据集上的实验结果表明：对StarGAN，所提方法的防御成功率分别达到99.80%和99.63%，生成的对抗图像的视觉质量相较于基于扩频对抗攻击的基准方法在结构相似性（SSIM）上分别提升了30.86%和26.63%，在峰值信噪比（PSNR）上分别提高了34.80%和36.15%。可见，所提方法可有效防御人脸伪造，同时提升对抗图像的视觉质量。

关键词: 人脸伪造, 主动防御, 注意力掩码, 对抗样本, 特征提取

CLC Number:

TP391

Yu WANG, Xianjin FANG, Gaoming YANG, Yifeng DING, Xinlu YANG. Active defense against face forgery based on attention mask and feature extraction[J]. Journal of Computer Applications, 2025, 45(3): 904-910.

王瑜, 方贤进, 杨高明, 丁一峰, 杨新露. 基于注意力掩码与特征提取的人脸伪造主动防御[J]. 《计算机应用》唯一官方网站, 2025, 45(3): 904-910.

Figures/Tables 8

References 26

1	周文柏，张卫明，俞能海，等. 人脸视频深度伪造与防御技术综述［J］. 信号处理， 2021， 37（12）： 2338-2355.
	ZHOU W B， ZHANG W M， YU N H， et al. An overview of Deepfake forgery and defense techniques ［J］. Journal of Signal Processing， 2021， 37（12）：2338-2355.
2	蔺琛皓，沈超，邓静怡，等. 虚假数字人脸内容生成与检测技术［J］. 计算机学报， 2023， 46（3）： 469-498.
	LIN C H， SHEN C， DENG J Y， et al. Digitally forged face content creation and detection ［J］. Chinese Journal of Computers， 2023， 46（3）： 469-498.
3	JUNG T， KIM S， KIM K. DeepVision： Deepfakes detection using human eye blinking pattern ［J］. IEEE Access， 2020， 8： 83144-83154.
4	MIRSKY Y， LEE W. The creation and detection of deepfakes： a survey ［J］. ACM Computing Surveys， 2022， 54（1）： No.7.
5	FU X， LI S， YUAN Y， et al. Forgery face detection via adaptive learning from multiple experts ［J］. Neurocomputing， 2023， 527： 110-118.
6	PUMAROLA A， AGUDO A， MARTINEZ A M， et al. GANimation： anatomically-aware facial animation from a single image ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11214. Cham： Springer， 2018： 835-851.
7	CHENG B， MISRA I， SCHWING A G， et al. Masked-attention mask Transformer for universal image segmentation ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 1280-1289.
8	NIE X， DUAN M， DING H， et al. Attention Mask R-CNN for ship detection and segmentation from remote sensing images ［J］. IEEE Access， 2020， 8： 9325-9334.
9	DOSOVITSKIY A， BROX T. Generating images with perceptual similarity metrics based on deep networks ［C］// Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2016： 658-666.
10	GATYS L A， ECKER A S， BETHGE M. Image style transfer using convolutional neural networks ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2414-2423.
11	JOHNSON J， ALAHI A， LI F F. Perceptual losses for real-time style transfer and super-resolution ［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9906. Cham： Springer， 2016： 694-711.
12	LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 105-114.
13	LIU S， DENG W. Very deep convolutional neural network based image classification using small training sample size ［C］// Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition. Piscataway： IEEE， 2015： 730-734.
14	LUAN F， PARIS S， SHECHTMAN E， et al. Deep photo style transfer ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6997-7005.
15	SITAULA C， HOSSAIN M B. Attention-based VGG16 model for COVID-19 chest X-ray image classification ［J］. Applied Intelligence， 2021， 51： 2850-2863.
16	TAMMINA S. Transfer learning using VGG16 with deep convolutional neural network for classifying images ［J］. International Journal of Scientific and Research Publications， 2019， 9（10）： 143-150.
17	FANG Z， YANG Y， LIN J， et al. Adversarial attacks for multi target image translation networks ［C］// Proceedings of the 2020 IEEE International Conference on Progress in Informatics and Computing. Piscataway： IEEE， 2020： 179-184.
18	HUANG H， WANG Y， CHEN Z， et al. CMUA-Watermark： a cross-model universal adversarial watermark for combating deepfakes ［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2022： 989-997.
19	QIU H， DU Y， LU T. The framework of cross-domain and model adversarial attack against deepfake ［J］. Future Internet， 2022， 14（2）： No.46.
20	RUIZ N， BARGAL S A， SCLAROFF S. Disrupting deepfakes： adversarial attacks against conditional image translation networks and facial manipulation systems ［C］// Proceedings of the 2020 European Conference on Computer Vision Workshops， LNCS 12538. Cham： Springer， 2020： 236-251.
21	WANG X， HUANG J， MA S， et al. DeepFake Disrupter： the detector of deepfake is my friend ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 14900-14909.
22	YEH C Y， CHEN H W， TSAI S L， et al. Disrupting image-translation-based DeepFake algorithms with adversarial attacks［C］// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision Workshops. Piscataway： IEEE， 2020： 53-62.
23	CHOI Y， CHOI M， KIM M， et al. StarGAN： unified generative adversarial networks for multi-domain image-to-image translation［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 8789-8797.
24	HUANG Q， ZHANG J， ZHOU W， et al. Initiative defense against facial manipulation ［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 1619-1627.
25	SHAHRIYAR S A， WRIGHT M. Evaluating robustness of sequence-based deepfake detector models by adversarial perturbation ［C］// Proceedings of the 1st Workshop on Security Implications of Deepfakes and Cheapfakes. New York： ACM， 2022： 13-18.
26	ZHU Y， CHEN Y， LI X， et al. Information-containing adversarial perturbation for combating facial manipulation systems ［J］. IEEE Transactions on Information Forensics and Security， 2023， 18： 2046-2059.

数据集	模型	Success/%	文献［20］方法			本文方法
数据集	模型	Success/%	L2	SSIM	PSNR/dB	L2	SSIM	PSNR/dB
CelebA	StarGAN	99.80	0.122 0	0.463 2	20.480 9	0.281 0	0.527 0	19.756 2
CelebA	GANimation	95.49	0.052 6	0.742 6	27.636 9	0.124 7	0.520 9	20.213 3
RaFD	StarGAN	99.63	0.057 9	0.628 8	24.180 1	0.379 6	0.531 7	21.755 1
RaFD	GANimation	96.01	0.095 6	0.649 5	30.803 6	0.387 4	0.486 7	18.498 9

数据集	模型	Success/%	文献［20］方法			本文方法
数据集	模型	Success/%	L2	SSIM	PSNR/dB	L2	SSIM	PSNR/dB
CelebA	StarGAN	99.80	0.122 0	0.463 2	20.480 9	0.281 0	0.527 0	19.756 2
CelebA	GANimation	95.49	0.052 6	0.742 6	27.636 9	0.124 7	0.520 9	20.213 3
RaFD	StarGAN	99.63	0.057 9	0.628 8	24.180 1	0.379 6	0.531 7	21.755 1
RaFD	GANimation	96.01	0.095 6	0.649 5	30.803 6	0.387 4	0.486 7	18.498 9

数据集	文献［20］方法			本文方法
数据集	L2	SSIM	PSNR/dB	L2	SSIM	PSNR/dB
CelebA	0.149 9	0.739 1	29.275 0	0.020 3	0.967 2	39.462 0
RaFD	0.094 4	0.653 7	25.773 1	0.038 6	0.827 8	35.090 0

数据集	文献［20］方法			本文方法
数据集	L2	SSIM	PSNR/dB	L2	SSIM	PSNR/dB
CelebA	0.149 9	0.739 1	29.275 0	0.020 3	0.967 2	39.462 0
RaFD	0.094 4	0.653 7	25.773 1	0.038 6	0.827 8	35.090 0

模型	本文方法（FGSM）		本文方法（I-FGSM）		本文方法（PGD）
模型	Success/%	L2	Success/%	L2	Success/%	L2
StarGAN	98.17	0.580 2	98.86	1.363 6	99.07	1.064 9
GANimation	93.24	0.094 2	95.08	0.240 6	96.98	0.207 3

Active defense against face forgery based on attention mask and feature extraction

基于注意力掩码与特征提取的人脸伪造主动防御

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 26

Related Articles 15

Recommended Articles

Metrics

baseline	Attention Mask	VGG16	SSIM	PSNR/dB
√			0.740 8	30.551 4
√	√		0.827 4	34.799 0
√		√	0.905 3	38.247 2
√	√	√	0.967 2	39.446 1

[1]	Huahua WANG, Zijian FAN, Ze LIU. Image adversarial example generation method based on multi-space probability enhancement [J]. Journal of Computer Applications, 2025, 45(3): 883-890.
[2]	Qiurun HE, Jie HU, Bo PENG, Tianyuan LI. Fabric defect detection algorithm based on context information and multi-scale feature fusion [J]. Journal of Computer Applications, 2025, 45(2): 640-646.
[3]	Tianqi ZHANG, Shuang TAN, Xiwen SHEN, Juan TANG. Image watermarking method combining attention mechanism and multi-scale feature [J]. Journal of Computer Applications, 2025, 45(2): 616-623.
[4]	Jietao LIANG, Bing LUO, Lanhui FU, Qingling CHANG, Nannan LI, Ningbo YI, Qi FENG, Xin HE, Fuqin DENG. Point cloud registration method based on coordinate geometric sampling [J]. Journal of Computer Applications, 2025, 45(1): 214-222.
[5]	Xin YANG, Xueni CHEN, Chunjiang WU, Shijie ZHOU. Short-term traffic flow prediction of urban highway based on variant residual model and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2947-2951.
[6]	Shuai FU, Xiaoying GUO, Ruyi BAI, Tao YAN, Bin CHEN. Age estimation method combining improved CloFormer model and ordinal regression [J]. Journal of Computer Applications, 2024, 44(8): 2372-2380.
[7]	Tong CHEN, Fengyu YANG, Yu XIONG, Hong YAN, Fuxing QIU. Construction method of voiceprint library based on multi-scale frequency-channel attention fusion [J]. Journal of Computer Applications, 2024, 44(8): 2407-2413.
[8]	Wudan LONG, Bo PENG, Jie HU, Ying SHEN, Danni DING. Road damage detection algorithm based on enhanced feature extraction [J]. Journal of Computer Applications, 2024, 44(7): 2264-2270.
[9]	Ruihua LIU, Zihe HAO, Yangyang ZOU. Gait recognition algorithm based on multi-layer refined feature fusion [J]. Journal of Computer Applications, 2024, 44(7): 2250-2257.
[10]	Jinfu WU, Yi LIU. Fast adversarial training method based on random noise and adaptive step size [J]. Journal of Computer Applications, 2024, 44(6): 1807-1815.
[11]	Zhihao WU, Ziqiu CHI, Ting XIAO, Zhe WANG. Meta-learning adaption for few-shot text-to-speech [J]. Journal of Computer Applications, 2024, 44(5): 1629-1635.
[12]	Yu ZHANG, Yan CHANG, Shibin ZHANG. Adversarial example detection algorithm based on quantum local intrinsic dimensionality [J]. Journal of Computer Applications, 2024, 44(2): 490-495.
[13]	Chenhui CUI, Suzhen LIN, Dawei LI, Xiaofei LU, Jie WU. Infrared dim small target tracking method based on Siamese network and Transformer [J]. Journal of Computer Applications, 2024, 44(2): 563-571.
[14]	Wenjie YAN, Dongyue DANG. Broad quantum state tomography model based on adaptive feature extraction [J]. Journal of Computer Applications, 2024, 44(12): 3861-3866.
[15]	Yifei SONG, Yi LIU. Fast adversarial training method based on data augmentation and label noise [J]. Journal of Computer Applications, 2024, 44(12): 3798-3807.