Detecting Text Manipulation in Images using Vision Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
Recent works have shown the effectiveness of Large Vision Language Models (VLMs or LVLMs) in image manipulation detection. However, text manipulation detection is largely missing in these studies. We bridge this knowledge gap by analyzing closed- and open-source VLMs on different text manipulation datasets. Our results suggest that open-source models are getting closer, but still behind closed-source ones like GPT- 4o. Additionally, we benchmark image manipulation detection-specific VLMs for text manipulation detection and show that they suffer from the generalization problem. We benchmark VLMs for manipulations done on in-the-wild scene texts and on fantasy ID cards, where the latter mimic a challenging real-world misuse.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 32 references that are in the map. First posted on arXiv 12 Sep 2025.
Builds on
22 of 32 references.
Closest ancestors · 2
Same direction, newest first
- 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model5 Dec
- 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models3 Oct
Other references · 20
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
- 2019On the Detection of Digital Face Manipulationcited by 260
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
- 2021MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detectioncited by 117
- 2021Image Manipulation Detection by Multi-View Multi-Scale Supervisioncited by 111
- 2023EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protectioncited by 111
- 2017Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)cited by 64
- 2023CRoSS: Diffusion Model Makes Controllable, Robust and Secure Image Steganographycited by 50
8 more
- 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 44
- 2024AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wildcited by 27
- 2024Revisiting Tampered Scene Text Detection in the Era of Generative AIcited by 21
- 2022Learning to Immunize Images for Tamper Localization and Self-Recoverycited by 19
- 2024First Competition on Presentation Attack Detection on ID Cardcited by 18
- 2025FoundPAD: Foundation Models Reloaded for Face Presentation Attack Detectioncited by 14
- 2025Exploring ChatGPT for Face Presentation Attack Detection in Zero and Few-Shot in-Context Learningcited by 12
- 2025FantasyID: A dataset for detecting digital manipulations of ID-documentscited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection
- 2025ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation
- 2026Rethinking VLMs for Image Forgery Detection and Localization
- 2024MVTamperBench: Evaluating Robustness of Vision-Language Models
- 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
- 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation
- 2025Text2VLM: Adapting Text-Only Datasets to Evaluate Alignment Training in Visual Language Models
- 2025Interpretable and Reliable Detection of AI-Generated Images via Grounded Reasoning in MLLMs