Data to 13 Oct 2026

LEGION: Learning to Ground and Explain for Synthetic Image Detection

arXiv 2503.15264 ↗First posted 19 Mar 2025cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

The rapid advancements in generative technology have emerged as a double-edged sword. While offering powerful tools that enhance convenience, they also pose significant social concerns. As defenders, current synthetic image detection methods often lack artifact-level textual interpretability and are overly focused on image manipulation detection, and current datasets usually suffer from outdated generators and a lack of fine-grained annotations. In this paper, we introduce SynthScars, a high-quality and diverse dataset consisting of 12,236 fully synthetic images with human-expert annotations. It features 4 distinct image content types, 3 categories of artifacts, and fine-grained annotations covering pixel-level segmentation, detailed textual explanations, and artifact category labels. Furthermore, we propose LEGION (LEarning to Ground and explain for Synthetic Image detectiON), a multimodal large language model (MLLM)-based image forgery analysis framework that integrates artifact detection, segmentation, and explanation. Building upon this capability, we further explore LEGION as a controller, integrating it into image refinement pipelines to guide the generation of higher-quality and more realistic images. Extensive experiments show that LEGION outperforms existing methods across multiple benchmarks, particularly surpassing the second-best traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score. Moreover, the refined images generated under its guidance exhibit stronger alignment with human preferences. The code, model, and dataset will be released.

Key facts. Cited by 65 papers in the map (data through 13 Oct 2026). Builds on 58 of its 73 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

58 of 73 references.

References · 58

2020Denoising Diffusion Probabilistic Modelscited by 20,952
2023GPT-4 Technical Reportcited by 18,278
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,713
2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,103
2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,484
2024GPT-4o System Cardcited by 4,536
2023Improved Baselines with Visual Instruction Tuningcited by 4,389
2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,323
2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,191
2021Improved Denoising Diffusion Probabilistic Modelscited by 3,751

38 more

2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
2017Axiomatic Attribution for Deep Networkscited by 3,368
2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,703
2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,604
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,546
2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,421
2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,360
2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,974
2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,012
2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 871
2019CNN-generated images are surprisingly easy to spot... for nowcited by 846
2023LISA: Reasoning Segmentation via Large Language Modelcited by 773
2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 733
2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 566
2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 503
2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 488
2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 448
2023GenImage: A Million-Scale Benchmark for Detecting AI-Generated Imagecited by 320
2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
2020What makes fake images detectable? Understanding properties that generalizecited by 273
2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
2024A Sanity Check for AI-generated Image Detectioncited by 187
2023Rich Human Feedback for Text-to-Image Generationcited by 141
2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
2019Detecting GAN generated Fake Images using Co-occurrence Matricescited by 124
2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Modelscited by 112
2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
2023CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Imagescited by 84
2022FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbationscited by 75
2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 64
2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistantcited by 60
2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
2021OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wildcited by 50
2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 44
2023Perceptual Artifacts Localization for Image Synthesis Taskscited by 43
2024A Single Simple Patch is All You Need for AI-generated Image Detectioncited by 43

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 29; 2026: 36.

65 papers.

Sort:

By area:

38 more

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way