# Rosetta: Large scale system for text detection and recognition in images

arXiv: https://arxiv.org/abs/1910.05085 · HTML: https://mlsanity.com/paper/1910.05085/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2019.

## Built on by (most cited first, 30 of 69)

- [TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text](https://mlsanity.com/paper/2105.05486/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [PromptCap: Prompt-Guided Task-Aware Image Captioning](https://mlsanity.com/paper/2211.09699/) (2022)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [Human-Adversarial Visual Question Answering](https://mlsanity.com/paper/2106.02280/) (2021)
- [Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer](https://mlsanity.com/paper/2311.13120/) (2023)
- [From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models](https://mlsanity.com/paper/2403.12027/) (2024)
- [MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding](https://mlsanity.com/paper/2108.06543/) (2021)
- [Multi-Granularity Prediction for Scene Text Recognition](https://mlsanity.com/paper/2209.03592/) (2022)
- [CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model](https://mlsanity.com/paper/2305.14014/) (2023)
- [MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning](https://mlsanity.com/paper/2112.06558/) (2021)
- [PreSTU: Pre-Training for Scene-Text Understanding](https://mlsanity.com/paper/2209.05534/) (2022)
- [Towards Unified Scene Text Spotting based on Sequence Generation](https://mlsanity.com/paper/2304.03435/) (2023)
- [Towards Accurate Text-based Image Captioning with Content Diversity Exploration](https://mlsanity.com/paper/2105.03236/) (2021)
- [On the Cross-dataset Generalization in License Plate Recognition](https://mlsanity.com/paper/2201.00267/) (2022)
- [Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literature](https://mlsanity.com/paper/2305.11033/) (2023)
- [OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese](https://mlsanity.com/paper/2305.04183/) (2023)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs](https://mlsanity.com/paper/2310.16033/) (2023)
- [Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling](https://mlsanity.com/paper/2108.08965/) (2021)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Towards Open-Set Text Recognition via Label-to-Prototype Learning](https://mlsanity.com/paper/2203.05179/) (2022)
- [Weakly-supervised Fingerspelling Recognition in British Sign Language Videos](https://mlsanity.com/paper/2211.08954/) (2022)
- [Leveraging Model Fusion for Improved License Plate Recognition](https://mlsanity.com/paper/2309.04331/) (2023)
- [What Large Language Models Bring to Text-rich VQA?](https://mlsanity.com/paper/2311.07306/) (2023)
- [ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting](https://mlsanity.com/paper/2403.00303/) (2024)
- [Open-set Text Recognition via Character-Context Decoupling](https://mlsanity.com/paper/2204.05535/) (2022)
- [Lumos : Empowering Multimodal LLMs with Scene Text Recognition](https://mlsanity.com/paper/2402.08017/) (2024)
- [Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition](https://mlsanity.com/paper/2307.13244/) (2023)

## Abstract (arXiv)

In this paper we present a deployed, scalable optical character recognition (OCR) system, which we call Rosetta, designed to process images uploaded daily at Facebook scale. Sharing of image content has become one of the primary ways to communicate information among internet users within social networks such as Facebook and Instagram, and the understanding of such media, including its textual information, is of paramount importance to facilitate search and recommendation applications. We present modeling techniques for efficient detection and recognition of text in images and describe Rosetta's system architecture. We perform extensive evaluation of presented technologies, explain useful practical approaches to build an OCR system at scale, and provide insightful intuitions as to why and how certain components work based on the lessons learnt during the development and deployment of the system.