Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline
Not on the map. Why?
Abstract (arXiv)
Existing facial forgery detection methods typically focus on binary classification or pixel-level localization, providing little semantic insight into the nature of the manipulation. To address this, we introduce Forgery Attribution Report Generation, a new multimodal task that jointly localizes forged regions ("Where") and generates natural language explanations grounded in the editing process ("Why"). This dual-focus approach goes beyond traditional forensics, providing a comprehensive understanding of the manipulation. To enable research in this domain, we present Multi-Modal Tamper Tracing (MMTT), a large-scale dataset of 152,217 samples, each with a process-derived ground-truth mask and a human-authored textual description, ensuring high annotation precision and linguistic richness. We further propose ForgeryTalker, a unified end-to-end framework that integrates vision and language via a shared encoder (image encoder + Q-former) and dual decoders for mask and text generation, enabling coherent cross-modal reasoning. Experiments show that ForgeryTalker achieves competitive performance on both report generation and forgery localization subtasks, i.e., 59.3 CIDEr and 73.67 IoU, respectively, establishing a baseline for explainable multimedia forensics. Dataset and code will be released to foster future research.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 33 of its 59 references that are in the map. First posted on arXiv 27 Dec 2024.
Builds on
33 of 59 references.
References · 33
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
- 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
- 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
- 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
21 more
- 2020DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detectioncited by 254
- 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
- 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
- 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
- 2020Media Forensics and DeepFakes: an overviewcited by 209
- 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
- 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
- 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 131
- 2024DF40: Toward Next-Generation Deepfake Detectioncited by 129
- 2019Recurrent Convolutional Strategies for Face Manipulation Detection in Videoscited by 118
- 2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistantcited by 61
- 2021OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wildcited by 50
- 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 44
- 2019GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detectioncited by 42
- 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 32
- 2024DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysiscited by 30
- 2024Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modelingcited by 14
- 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMscited by 13
- 2021An Experimental Evaluation on Deepfake Detection using Deep Face Recognitioncited by 12
- 2024GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learningcited by 12
- 2022Face Forgery Detection Based on Facial Region Displacement Trajectory Seriescited by 4
Built on by
Citing papers per year: 2025: 5; 2026: 1.
6 papers.
- 2025BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanationcited by 33
- 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLMcited by 12
- 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detectioncited by 8
- 2025FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adaptercited by 7
- 2025AnimeDL-2M: Million-Scale AI-Generated Anime Image Detection and Localization in Diffusion Eracited by 4
- 2026Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detectioncited by 2
- 2026Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection2 Mar 2026
- 2025FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adapter27 Aug 2025
- 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detection6 Aug 2025
- 2025BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanation19 May 2025
- 2025AnimeDL-2M: Million-Scale AI-Generated Anime Image Detection and Localization in Diffusion Era15 Apr 2025
- 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLM7 Feb 2025