Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (5): 1571-1577.DOI: 10.11772/j.issn.1001-9081.2022040568

• Multimedia computing and computer simulation • Previous Articles    

Unsupervised face forgery video detection based on reconstruction error

Zhe XU, Zhihong WANG, Cunyu SHAN, Yaru SUN, Ying YANG()   

  1. Research and Development Base of Cyberspace Security Technology,The Third Research Institute of The Ministry of Public Security,Shanghai 200031,China
  • Received:2022-04-24 Revised:2022-06-17 Accepted:2022-06-17 Online:2022-07-11 Published:2023-05-10
  • Contact: Ying YANG
  • About author:XU Zhe, born in 1993, M. S., research probationer. His research interests include natural language processing, temporal anomaly detection, face forgery detection.
    WANG Zhihong, born in 1990, Ph. D., assistant research fellow. His research interests include natural language processing, event mining, network public hazards management.
    SHAN Cunyu, born in 1993, M. S., research probationer. His research interests include image recognition, data mining.
    SUN Yaru, born in 1993, M. S., research probationer. Her research interests include natural language processing, data mining.
    YANG Ying, born in 1981, Ph. D., associate research fellow. Her research interests include big data analysis, information security.
  • Supported by:
    National Key Research and Development Program of China(2021YFB3101405)

基于重构误差的无监督人脸伪造视频检测

许喆, 王志宏, 单存宇, 孙亚茹, 杨莹()   

  1. 公安部第三研究所 网络空间安全技术研发基地,上海 200031
  • 通讯作者: 杨莹
  • 作者简介:许喆(1993—),男,安徽滁州人,研究实习员,硕士,CCF会员,主要研究方向:自然语言处理、时序异常检测、人脸伪造检测
    王志宏(1990—),男,江苏泰兴人,助理研究员,博士,CCF会员,主要研究方向:自然语言处理、事件挖掘、网络公害治理
    单存宇(1993—),男,江苏大丰人,研究实习员,硕士,CCF会员,主要研究方向:图像识别、数据挖掘
    孙亚茹(1993—),女,山东菏泽人,研究实习员,硕士,CCF会员,主要研究方向:自然语言处理、数据挖掘
    杨莹(1981—),女,河南商丘人,副研究员,博士,CCF会员,主要研究方向:大数据分析、信息安全。yangying@mcst.org.cn
  • 基金资助:
    国家重点研发计划项目(2021YFB3101405)

Abstract:

The current supervised face forgery video detection methods need a large amount of labeled data. In order to solve the practical problems of fast iteration and many kinds of video forgery methods, the unsupervised idea in temporal anomaly detection was introduced into face forgery video detection, the face forgery video detection task was transformed into unsupervised video anomaly detection task, and an unsupervised face forgery video detection method based on reconstruction error was proposed. Firstly, the facial landmark sequence of continuous frames in the video to be detected was extracted. Secondly, the facial landmark sequence in the video to be detected was reconstructed based on multi-granularity information such as deviation features, local features and temporal features. Thirdly, the reconstruction error between the original sequence and the reconstructed sequence was calculated. Finally, the score was calculated according to the peak frequency of the reconstruction error to detect the forgery video automatically. Experimental results show that compared with detection methods such as LRNet (Landmark Recurrent Network) and Xception-c23, the proposed method has the AUC (Area Under Curve) of the detection performance increased by up to 27.6%, and the AUC of the transplantation performance increased by 30.4%.

Key words: face forgery detection, unsupervised learning, temporal anomaly detection, generative model, facial landmark

摘要:

目前有监督的人脸伪造视频检测方法需要大量标注数据。为解决视频伪造方法迭代快、种类多等现实问题,将时序异常检测中的无监督思想引入人脸伪造视频检测,将伪造视频检测任务转为无监督的视频异常检测任务,提出一种基于重构误差的无监督人脸伪造视频检测模型。首先,抽取待检测视频中连续帧的人脸特征点序列;其次,基于偏移特征、局部特征、时序特征等多粒度信息对待检测视频中人脸特征点序列进行重构;然后,计算原始序列与重构序列之间的重构误差;最后,根据重构误差的波峰频率计算得分对伪造视频进行自动检测。实验结果表明,在FaceShifter、FaceSwap等人脸视频伪造方法上,与LRNet (Landmark Recurrent Network)、Xception-c23等检测方法相比,所提方法的检测性能的曲线下方面积(AUC)最多增加了27.6%,移植性能的AUC最多增加了30.4%。

关键词: 人脸伪造检测, 无监督学习, 时序异常检测, 生成模型, 人脸特征点

CLC Number: