# LEGION: Learning to Ground and Explain for Synthetic Image Detection

arXiv: https://arxiv.org/abs/2503.15264 · HTML: https://mlsanity.com/paper/2503.15264/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 73 references that are in the map.
- First posted on arXiv 19 Mar 2025.

## Builds on (most cited first, 30 of 58)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image](https://mlsanity.com/paper/2306.08571/) (2023)

## Built on by (most cited first, 30 of 65)

- [AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models](https://mlsanity.com/paper/2507.02664/) (2025)
- [Shifting AI Efficiency From Model-Centric to Data-Centric Compression](https://mlsanity.com/paper/2505.19147/) (2025)
- [BusterX++: Towards Unified Cross-Modal AI-Generated Content Detection and Explanation with MLLM](https://mlsanity.com/paper/2507.14632/) (2025)
- [Skyra: AI-Generated Video Detection via Grounded Artifact Reasoning](https://mlsanity.com/paper/2512.15693/) (2025)
- [AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs](https://mlsanity.com/paper/2510.07293/) (2025)
- [Toward Generalizable Forgery Detection and Reasoning](https://mlsanity.com/paper/2503.21210/) (2025)
- [Seeing Before Reasoning: A Unified Framework for Generalizable and Explainable Fake Image Detection](https://mlsanity.com/paper/2509.25502/) (2025)
- [Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods](https://mlsanity.com/paper/2510.07143/) (2025)
- [RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewards](https://mlsanity.com/paper/2512.00473/) (2025)
- [UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization](https://mlsanity.com/paper/2510.03161/) (2025)
- [BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception](https://mlsanity.com/paper/2510.09361/) (2025)
- [RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detection](https://mlsanity.com/paper/2508.04524/) (2025)
- [Semantic Visual Anomaly Detection and Reasoning in AI-Generated Images](https://mlsanity.com/paper/2510.10231/) (2025)
- [Venus-DeFakerOne: Unified Fake Image Detection & Localization](https://mlsanity.com/paper/2605.14091/) (2026)
- [Guard Me If You Know Me: Protecting Specific Face-Identity from Deepfakes](https://mlsanity.com/paper/2505.19582/) (2025)
- [ThinkFake: Reasoning in Multimodal Large Language Models for AI-Generated Image Detection](https://mlsanity.com/paper/2509.19841/) (2025)
- [AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detection](https://mlsanity.com/paper/2505.15173/) (2025)
- [DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLM](https://mlsanity.com/paper/2512.10619/) (2025)
- [EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection](https://mlsanity.com/paper/2603.17343/) (2026)
- [So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection](https://mlsanity.com/paper/2505.18660/) (2025)
- [IPCV: Information-Preserving Compression for MLLM Visual Encoders](https://mlsanity.com/paper/2512.18747/) (2025)
- [Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Images](https://mlsanity.com/paper/2510.04225/) (2025)
- [AlignGemini: Generalizable AI-Generated Image Detection Through Task-Model Alignment](https://mlsanity.com/paper/2512.06746/) (2025)
- [TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection](https://mlsanity.com/paper/2512.10652/) (2025)
- [Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection](https://mlsanity.com/paper/2512.16300/) (2025)
- [FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detection](https://mlsanity.com/paper/2605.30062/) (2026)
- [Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detection](https://mlsanity.com/paper/2601.19430/) (2026)
- [UniGenDet: A Unified Generative-Discriminative Framework for Co-Evolutionary Image Generation and Generated Image Detection](https://mlsanity.com/paper/2604.21904/) (2026)
- [Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection](https://mlsanity.com/paper/2605.01638/) (2026)
- [ReAlign: Generalizable Image Forgery Detection via Reasoning-Aligned Representation](https://mlsanity.com/paper/2605.16080/) (2026)

## Abstract (arXiv)

The rapid advancements in generative technology have emerged as a double-edged sword. While offering powerful tools that enhance convenience, they also pose significant social concerns. As defenders, current synthetic image detection methods often lack artifact-level textual interpretability and are overly focused on image manipulation detection, and current datasets usually suffer from outdated generators and a lack of fine-grained annotations. In this paper, we introduce SynthScars, a high-quality and diverse dataset consisting of 12,236 fully synthetic images with human-expert annotations. It features 4 distinct image content types, 3 categories of artifacts, and fine-grained annotations covering pixel-level segmentation, detailed textual explanations, and artifact category labels. Furthermore, we propose LEGION (LEarning to Ground and explain for Synthetic Image detectiON), a multimodal large language model (MLLM)-based image forgery analysis framework that integrates artifact detection, segmentation, and explanation. Building upon this capability, we further explore LEGION as a controller, integrating it into image refinement pipelines to guide the generation of higher-quality and more realistic images. Extensive experiments show that LEGION outperforms existing methods across multiple benchmarks, particularly surpassing the second-best traditional expert on SynthScars by 3.31% in mIoU and 7.75% in F1 score. Moreover, the refined images generated under its guidance exhibit stronger alignment with human preferences. The code, model, and dataset will be released.