mlsanityData to 7 Oct 2026

Ocean-OCR: Towards General OCR Application via a Vision-Language Model

arXiv 2501.15558 ↗First posted 26 Jan 2025cs.CVcited by 24

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Multimodal large language models (MLLMs) have shown impressive capabilities across various domains, excelling in processing and understanding information from multiple modalities. Despite the rapid progress made previously, insufficient OCR ability hinders MLLMs from excelling in text-related tasks. In this paper, we present Ocean-OCR, a 3B MLLM with state-of-the-art performance on various OCR scenarios and comparable understanding ability on general tasks. We employ Native Resolution ViT to enable variable resolution input and utilize a substantial collection of high-quality OCR datasets to enhance the model performance. We demonstrate the superiority of Ocean-OCR through comprehensive experiments on open-source OCR benchmarks and across various OCR scenarios. These scenarios encompass document understanding, scene text recognition, and handwritten recognition, highlighting the robust OCR capabilities of Ocean-OCR. Note that Ocean-OCR is the first MLLM to outperform professional OCR models such as TextIn and PaddleOCR.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 67 references that are in the map. First posted on arXiv 26 Jan 2025.

Builds on

27 of 67 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document7 Mar
  2. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding4 Jul
  3. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models13 May

Other references · 24

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2023A Survey on Multimodal Large Language Modelscited by 898
  11. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  12. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 235
12 more
  1. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  2. 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 156
  3. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
  4. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
  5. 2024DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perceptioncited by 72
  6. 2021PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR Systemcited by 39
  7. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  8. 2024BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devicescited by 25
  9. 2024POINTS1.5: Building a Vision-Language Model towards Real World Applicationscited by 12
  10. 2024BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baselinecited by 6
  11. 2015Incidental Scene Text Understanding: Recent Progresses on ICDAR 2015 Robust Reading Competition Challenge 4cited by 2
  12. 2024DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective Partitioningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 10.

24 papers.

Sort:

By area:

  1. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 84
  2. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
  3. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 26
  4. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 20
  5. 2025AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Paperscited by 18
  6. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  7. 2025VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Modelscited by 14
  8. 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMscited by 12
  9. 2025When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understandingcited by 12
  10. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 12
  11. 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wildcited by 11
  12. 2025Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Modelscited by 10
12 more
  1. 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Modelcited by 10
  2. 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translationcited by 8
  3. 2025Reasoning-Aligned Perception Decoupling for Scalable Multi-modal Reasoningcited by 7
  4. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  5. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decodingcited by 5
  6. 2026RTPrune: Reading-Twice Inspired Token Pruning for Efficient DeepSeek-OCR Inferencecited by 4
  7. 2026OmniOCR: Generalist OCR for Ethnic Minority Languagescited by 2
  8. 2026Geometric Risk Control for Vision-Language Model OCRcited by 1
  9. 2025Bridging Vision, Language, and Mathematics: Pictographic Character Reconstruction with B'ezier Curvescited by 0
  10. 2026Evaluating Vision-Language Models as a Zero-Shot Learning Alternative to You Only Look Once and Optical Character Recognition for Nigerian License Plate Recognitioncited by 0
  11. 2026HPD-Parsing: Hierarchical Parallel Document Parsingcited by 0
  12. 2026SupGRPO: Enhancing GRPO with Matching-based Online SFT for Text Spottingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PP-OCRv5: A Specialized 5M-Parameter Model Rivaling Billion-Parameter Vision-Language Models on OCR Tasks
  2. 2026Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models
  3. 2025Context-Independent OCR with Multimodal LLMs: Effects of Image Resolution and Visual Complexity
  4. 2025MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios
  5. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning
  6. 2024TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models
  7. 2026LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
  8. 2026From Retrieval to Recognition:How Vision--Language Models Become OCR Specialists