mlsanityData to 14 Oct 2026

Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline

arXiv 2412.19685 ↗First posted 27 Dec 2024cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Existing facial forgery detection methods typically focus on binary classification or pixel-level localization, providing little semantic insight into the nature of the manipulation. To address this, we introduce Forgery Attribution Report Generation, a new multimodal task that jointly localizes forged regions ("Where") and generates natural language explanations grounded in the editing process ("Why"). This dual-focus approach goes beyond traditional forensics, providing a comprehensive understanding of the manipulation. To enable research in this domain, we present Multi-Modal Tamper Tracing (MMTT), a large-scale dataset of 152,217 samples, each with a process-derived ground-truth mask and a human-authored textual description, ensuring high annotation precision and linguistic richness. We further propose ForgeryTalker, a unified end-to-end framework that integrates vision and language via a shared encoder (image encoder + Q-former) and dual decoders for mask and text generation, enabling coherent cross-modal reasoning. Experiments show that ForgeryTalker achieves competitive performance on both report generation and forgery localization subtasks, i.e., 59.3 CIDEr and 73.67 IoU, respectively, establishing a baseline for explainable multimedia forensics. Dataset and code will be released to foster future research.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 33 of its 59 references that are in the map. First posted on arXiv 27 Dec 2024.

Builds on

33 of 59 references.

References · 33

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  10. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  11. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
  12. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
21 more
  1. 2020DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detectioncited by 254
  2. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
  3. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  4. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
  5. 2020Media Forensics and DeepFakes: an overviewcited by 209
  6. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  7. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  8. 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 131
  9. 2024DF40: Toward Next-Generation Deepfake Detectioncited by 129
  10. 2019Recurrent Convolutional Strategies for Face Manipulation Detection in Videoscited by 118
  11. 2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistantcited by 61
  12. 2021OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wildcited by 50
  13. 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 44
  14. 2019GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detectioncited by 42
  15. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 32
  16. 2024DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysiscited by 30
  17. 2024Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modelingcited by 14
  18. 2024Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMscited by 13
  19. 2021An Experimental Evaluation on Deepfake Detection using Deep Face Recognitioncited by 12
  20. 2024GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learningcited by 12
  21. 2022Face Forgery Detection Based on Facial Region Displacement Trajectory Seriescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.