OCR-Agent: Agentic OCR with Capability and Memory Reflection
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
also in 1 more
Abstract (arXiv)
Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs' reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 21 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
18 of 21 references.
Closest ancestors · 3
Same direction, newest first
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
- 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning31 Dec
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models13 May
Other references · 15
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
- 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
- 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 278
- 2018TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapescited by 114
- 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
3 more
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026From Retrieval to Recognition:How Vision--Language Models Become OCR Specialists
- 2026Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics
- 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning
- 2024An Intelligent Agentic System for Complex Image Restoration Problems
- 2026Beyond Blind Compliance: Benchmarking Task Verification in OCR Reasoning
- 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model
- 2026OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding
- 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models