mlsanityData to 7 Oct 2026

COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images

arXiv 1601.07140 ↗First posted 26 Jan 2016cs.CVcited by 235

Not on the map. Why?

Cited by 55 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper describes the COCO-Text dataset. In recent years large-scale datasets like SUN and Imagenet drove the advancement of scene understanding and object recognition. The goal of COCO-Text is to advance state-of-the-art in text detection and recognition in natural images. The dataset is based on the MS COCO dataset, which contains images of complex everyday scenes. The images were not collected with text in mind and thus contain a broad variety of text instances. To reflect the diversity of text in natural scenes, we annotate text with (a) location in terms of a bounding box, (b) fine-grained classification into machine printed text and handwritten text, (c) classification into legible and illegible text, (d) script of the text and (e) transcriptions of legible text. The dataset contains over 173k text annotations in over 63k images. We provide a statistical analysis of the accuracy of our annotations. In addition, we present an analysis of three leading state-of-the-art photo Optical Character Recognition (OCR) approaches on our dataset. While scene text detection and recognition enjoys strong advances in recent years, we identify significant shortcomings motivating future work.

Key facts. Cited by 235 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jan 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 20; 2022: 39; 2023: 47; 2024: 63; 2025: 45; 2026: 20.

235 papers.

Sort:

By area:

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  2. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  3. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  4. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  5. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  6. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  7. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  8. 2024An Introduction to Vision-Language Modelingcited by 175
  9. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 174
  10. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  11. 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 170
  12. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
38 more
  1. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 148
  2. 2023PACO: Parts and Attributes of Common Objectscited by 146
  3. 2023ImageBind-LLM: Multi-modality Instruction Tuningcited by 142
  4. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  5. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  6. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  7. 2022MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuningcited by 109
  8. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 100
  9. 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 98
  10. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  11. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 88
  12. 2021ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spottingcited by 84
  13. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  14. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  15. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  16. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  17. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  18. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  19. 2021PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Textcited by 65
  20. 2023ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuningcited by 65
  21. 2021What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labelscited by 63
  22. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  23. 2023Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Modelscited by 59
  24. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 58
  25. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  26. 2023Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizercited by 53
  27. 2025FineVision: Open Data Is All You Needcited by 51
  28. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 49
  29. 2024MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understandingcited by 46
  30. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  31. 2022Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognitioncited by 45
  32. 2022Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognitioncited by 45
  33. 2023Revisiting Scene Text Recognition: A Data Perspectivecited by 45
  34. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  35. 2024Small Language Model Meets with Reinforced Vision Vocabularycited by 43
  36. 2024VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Modelscited by 42
  37. 2024SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Foldingcited by 42
  38. 2022Towards Weakly-Supervised Text Spotting using a Multi-Task Transformercited by 41

50 most cited of 235 All 235 (JSON)