mlsanityData to 14 Oct 2026

From Sharp Eyes to Expert Mind: Internalizing Expert Knowledge in MLLMs for Tampered Text Detection

arXiv 2609.36145 ↗First posted 28 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Tampered Text Detection (TTD) is essential for safeguarding document authenticity in security-critical workflows. Existing expert models are effective at capturing subtle manipulation traces but often generalize poorly across diverse document domains, while Multimodal Large Language Models (MLLMs) offer stronger semantic understanding and transferability yet remain insensitive to fine-grained forensic artifacts. This complementarity motivates us to investigate how expert forensic perception can be internalized into an MLLM rather than merely accessed through an external module. We identify a fundamental Double Mismatch that hinders this goal: a Spatial Precision Mismatch between coarse visual tokens and tiny tampered regions, and a Perceptual Granularity Mismatch between semantics-oriented pre-training and low-level forensic perception. To address these challenges, we propose Expert Knowledge Internalization (EKI), a progressive two-stage framework that transfers forensic expertise into the MLLM itself. In Stage 1, Text-Focused and Image-Focused strategies establish precise spatial focus on small text regions. In Stage 2, the proposed Forensic-General Representation Alignment (FGRA) loss aligns shallow LLM representations with those of a pre-trained forensic expert, enabling the model to acquire fine-grained artifact perception before such cues are diluted by deeper semantic abstraction. Extensive experiments on multiple in-domain and cross-domain benchmarks demonstrate that EKI achieves state-of-the-art performance and stronger generalization than existing expert-model-based and MLLM-based methods. Moreover, the expert is required only during training, allowing the resulting MLLM to maintain inference efficiency nearly identical to the vanilla model without relying on any external expert at inference.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 55 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

35 of 55 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2019Character Region Awareness for Text Detectioncited by 278
  9. 2018Learning Rich Features for Image Manipulation Detectioncited by 154
  10. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  11. 2025PaddleOCR 3.0 Technical Reportcited by 143
  12. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 134
23 more
  1. 2021PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localizationcited by 133
  2. 2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
  3. 2024Common Sense Reasoning for Deepfake Detectioncited by 65
  4. 2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistantcited by 61
  5. 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 44
  6. 2024IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localizationcited by 42
  7. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 41
  8. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  9. 2024X2-DFD: A framework for eXplainable and eXtendable Deepfake Detectioncited by 35
  10. 2024Mesoscopic Insights: Orchestrating Multi-scale & Hybrid Architecture for Image Manipulation Localizationcited by 35
  11. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  12. 2024Can We Get Rid of Handcrafted Feature Extractors? SparseViT: Nonsemantics-Centered, Parameter-Efficient Image Manipulation Localization through Spare-Coding Transformercited by 29
  13. 2024LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understandingcited by 24
  14. 2025ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localizationcited by 22
  15. 2024Revisiting Tampered Scene Text Detection in the Era of Generative AIcited by 21
  16. 2023Toward Real Text Manipulation Detection: New Dataset and New Solutioncited by 20
  17. 2025Seeing Before Reasoning: A Unified Framework for Generalizable and Explainable Fake Image Detectioncited by 20
  18. 2025Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuningcited by 11
  19. 2022Towards Effective Image Manipulation Detection with Proposal Contrastive Learningcited by 8
  20. 2024TextSleuth: Towards Explainable Tampered Text Detectioncited by 8
  21. 2022ReLoc: A Restoration-Assisted Framework for Robust Image Tampering Localizationcited by 5
  22. 2025Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesiscited by 3
  23. 2026Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TextShield-R1: Reinforced Reasoning for Tampered Text Detection
  2. 2026Team MSU GenText-Forensics Challenge 2026 Technical Report
  3. 2025Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline
  4. 2025Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localization
  5. 2026ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
  6. 2025Can GPT tell us why these images are synthesized? Empowering Multimodal Large Language Models for Forensics
  7. 2026DocShield: Towards AI Document Safety via Evidence-Grounded Agentic Reasoning
  8. 2026Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics