mlsanityData to 14 Oct 2026

Can VLMs Detect and Localize Fine-Grained AI-Edited Images?

arXiv 2505.15644 ↗First posted 21 May 2025cs.CV cs.AI cs.CRcited by 4

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Fine-grained detection and localization of localized image edits is crucial for assessing content authenticity, especially as modern diffusion models and image editors can produce highly realistic manipulations. However, this problem faces three key challenges: (1) most AIGC detectors produce only a global real-or-fake label without indicating where edits occur; (2) traditional computer vision methods for edit localization typically rely on costly pixel-level annotations; and (3) there is no large-scale, modern benchmark specifically targeting edited-image detection. To address these gaps, we develop an automated data-generation pipeline and construct FragFake, a large-scale benchmark of AI-edited images spanning multiple source datasets, diverse editing models, and several common edit types. Building on FragFake, we are the first to systematically study vision language models (VLMs) for edited-image classification and edited-region localization. Our experiments show that pretrained VLMs, including GPT4o, perform poorly on this task, whereas fine-tuned models such as Qwen2.5-VL achieve high accuracy and substantially higher object precision across all settings. We further explore GRPO-based RLVR training, which yields modest metric gains while improving the interpretability of model outputs. Ablation and transfer analyses reveal how data balancing, training size, LoRA rank, and training domain affect performance, and highlight both the potential and the limitations of cross-editor and cross-dataset generalization. We anticipate that this work will establish a solid foundation to facilitate and inspire subsequent research endeavors in the domain of multimodal content authenticity.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 31 of its 44 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

31 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Diffusion Model-Based Image Editing: A Survey27 Feb
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  3. 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models13 Oct

Other references · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2023Segment Anythingcited by 8,234
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2016Densely Connected Convolutional Networkscited by 6,720
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  11. 2025Qwen2.5-VL Technical Reportcited by 5,587
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
16 more
  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2022A ConvNet for the 2020scited by 3,579
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  7. 2023LIMA: Less Is More for Alignmentcited by 1,100
  8. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  9. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  10. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  11. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 104
  12. 2024Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Mediacited by 35
  13. 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Modelscited by 24
  14. 2023Rethinking Image Editing Detection in the Era of Generative AI Revolutioncited by 21
  15. 2024Organic or Diffused: Can We Distinguish Human Art from AI-generated Images?cited by 21
  16. 2024Blessing or curse? A survey on the Impact of Generative AI on Fake Newscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baseline
  2. 2025X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models
  3. 2026Rethinking VLMs for Image Forgery Detection and Localization
  4. 2026FineEdit: Fine-Grained Image Edit with Bounding Box Guidance
  5. 2024EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM
  6. 2025PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models
  7. 2026Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editing
  8. 2026Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis