Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Construction and benchmark detection of multimodal partial forgery dataset
Shengyou ZHENG, Yanxiang CHEN, Zuxing ZHAO, Haiyang LIU
Journal of Computer Applications    2024, 44 (10): 3134-3140.   DOI: 10.11772/j.issn.1001-9081.2023101506
Abstract92)   HTML3)    PDF (1323KB)(27)       Save

Aiming at the lack of multimodal forgery scenarios and partial forgery scenarios in existing video forgery datasets, a multimodal partial forgery dataset with adjustable forgery ratios — PartialFAVCeleb was constructed by using a wide varieties of audio and video forgery methods. The proposed dataset was based on the FakeAVCeleb multimodal forgery dataset and was with the real and forged data spliced, in which the forgery data were generated by four methods, that is, FaceSwap, FSGAN (Face Swapping Generative Adversarial Network), Wav2Lip (Wave to Lip), and SV2TTS (Speaker Verification to Text-To-Speech). In the splicing process, probabilistic methods were used to generate the locations of the forgery segments in the time domain and modality, then the boundary was randomized to fit the actual forged scenario. And, the phenomenon of background hopping was avoided through material screening. The finally obtained dataset contains forgery videos of different ratios, with one ratio corresponding to 3 970 video data. In the benchmark detection, several audio and video feature extractors were used. And the data was tested in strong supervised and weakly-supervised conditions respectively, and Hierarchical Multi-Instance Learning (HMIL) method was used to realize the latter condition. As the test results indicate, for each test model, the performance on data with low forgery ratio is significantly inferior to that on data with high forgery ratio, and the performance under weakly-supervised condition is significantly inferior to that under strong supervised condition. The difficulty of weakly-supervised detection of proposed partial forgery dataset is verified. Experimental results show that the multimodal partial forgery scenario represented by the proposed dataset has sufficient research value.

Table and Figures | Reference | Related Articles | Metrics