mlsanityData to 7 Oct 2026

ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation

arXiv 2608.20122 ↗First posted 20 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Large multimodal models (LMMs) have demonstrated strong OCR recognition capabilities, yet remain vulnerable to adversarial visual text that is readable to humans but challenging for models to localize and recognize. Existing OCR benchmarks mainly focus on natural or document-style text, while adversarial OCR evaluations remain limited in scale, task coverage, or region-aware evaluation. In this paper, we formulate adversarial OCR as a grounded OCR perception task and introduce AdvSpot, the first benchmark for grounded adversarial OCR evaluation. AdvSpot comprises 390 images with region-level annotations, spanning 5 primary categories and 13 fine-grained adversarial OCR types. To address this challenge, we propose ArmorOCR, a two-stage training framework for robust adversarial OCR perception. ArmorOCR first acquires missing adversarial OCR perception from privileged transformed observations through On-Policy Self-Distillation (OPSD), and then refines grounded OCR perception through Group Relative Policy Optimization (GRPO) with task-conditioned rewards for localization, recognition, full spotting, and visual question answering (VQA). Experiments on our AdvSpot, other adversarial OCR benchmarks, and general OCR benchmarks demonstrate that ArmorOCR consistently improves adversarial OCR perception while preserving competitive general OCR capability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 29 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

20 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation8 Apr

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2025Qwen3-VL Technical Reportcited by 2,534
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  6. 2019Towards VQA Models That Can Readcited by 1,696
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  9. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  10. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  11. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  12. 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 80
7 more
  1. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 71
  2. 2026Self-Distilled Agentic Reinforcement Learningcited by 49
  3. 2025CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoningcited by 16
  4. 2026ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillationcited by 12
  5. 2026Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillationcited by 9
  6. 2025VACoT: Rethinking Visual Data Augmentation with VLMscited by 8
  7. 2025SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinkingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Models
  2. 2026Detecting Adversarial Images through Response Profiles of Vision-Language Models
  3. 2026Adversarial Attacks on Deep OCR Systems
  4. 2024A Unified Understanding of Adversarial Vulnerability Regarding Unimodal Models and Vision-Language Pre-training Models
  5. 2026Reading Between the Pixels: Linking Text-Image Embedding Alignment to Typographic Attack Success on Vision-Language Models
  6. 2024SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments
  7. 2024Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective
  8. 2025Attention-Guided Patch-Wise Sparse Adversarial Attacks on Vision-Language-Action Models