Can VLMs Detect and Localize Fine-Grained AI-Edited Images?
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Fine-grained detection and localization of localized image edits is crucial for assessing content authenticity, especially as modern diffusion models and image editors can produce highly realistic manipulations. However, this problem faces three key challenges: (1) most AIGC detectors produce only a global real-or-fake label without indicating where edits occur; (2) traditional computer vision methods for edit localization typically rely on costly pixel-level annotations; and (3) there is no large-scale, modern benchmark specifically targeting edited-image detection. To address these gaps, we develop an automated data-generation pipeline and construct FragFake, a large-scale benchmark of AI-edited images spanning multiple source datasets, diverse editing models, and several common edit types. Building on FragFake, we are the first to systematically study vision language models (VLMs) for edited-image classification and edited-region localization. Our experiments show that pretrained VLMs, including GPT4o, perform poorly on this task, whereas fine-tuned models such as Qwen2.5-VL achieve high accuracy and substantially higher object precision across all settings. We further explore GRPO-based RLVR training, which yields modest metric gains while improving the interpretability of model outputs. Ablation and transfer analyses reveal how data balancing, training size, LoRA rank, and training domain affect performance, and highlight both the potential and the limitations of cross-editor and cross-dataset generalization. We anticipate that this work will establish a solid foundation to facilitate and inspire subsequent research endeavors in the domain of multimodal content authenticity.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 31 of its 44 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
31 of 44 references.
Closest ancestors · 3
Same direction, newest first
- 2024Diffusion Model-Based Image Editing: A Survey27 Feb
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
- 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models13 Oct
Other references · 28
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023Segment Anythingcited by 8,234
- 2023Visual Instruction Tuningcited by 7,869
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
16 more
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2022A ConvNet for the 2020scited by 3,579
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Gemma 3 Technical Reportcited by 1,824
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 104
- 2024Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Mediacited by 35
- 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Modelscited by 24
- 2023Rethinking Image Editing Detection in the Era of Generative AI Revolutioncited by 21
- 2024Organic or Diffused: Can We Distinguish Human Art from AI-generated Images?cited by 21
- 2024Blessing or curse? A survey on the Impact of Generative AI on Fake Newscited by 12
Built on by
Citing papers per year: 2025: 3; 2026: 1.
4 papers.
- 2025ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanationcited by 5
- 2025Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Drivingcited by 4
- 2025DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detectioncited by 3
- 2026Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectificationcited by 1
- 2026Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification25 Mar 2026
- 2025DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detection24 Nov 2025
- 2025ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation18 Nov 2025
- 2025Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Driving3 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2025DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baseline
- 2025X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models
- 2026Rethinking VLMs for Image Forgery Detection and Localization
- 2026FineEdit: Fine-Grained Image Edit with Bounding Box Guidance
- 2024EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM
- 2025PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models
- 2026Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editing
- 2026Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis