mlsanityData to 7 Oct 2026

OCR-Agent: Agentic OCR with Capability and Memory Reflection

arXiv 2602.21053 ↗First posted 24 Feb 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs' reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 21 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

18 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
  2. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning31 Dec
  3. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models13 May

Other references · 15

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  7. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  8. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
  9. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
  10. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 278
  11. 2018TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapescited by 114
  12. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
3 more
  1. 2018Chargrid: Towards Understanding 2D Documentscited by 100
  2. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 66
  3. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Retrieval to Recognition:How Vision--Language Models Become OCR Specialists
  2. 2026Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics
  3. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning
  4. 2024An Intelligent Agentic System for Complex Image Restoration Problems
  5. 2026Beyond Blind Compliance: Benchmarking Task Verification in OCR Reasoning
  6. 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model
  7. 2026OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding
  8. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models