# Efficient OCR for Building a Diverse Digital History

arXiv: https://arxiv.org/abs/2304.02737 · HTML: https://mlsanity.com/paper/2304.02737/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 64 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 30 of 38)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [Revisiting the Sibling Head in Object Detector](https://mlsanity.com/paper/2003.07540/) (2020)
- [Sequence-to-Sequence Contrastive Learning for Text Recognition](https://mlsanity.com/paper/2012.10873/) (2020)
- [Convolutional Character Networks](https://mlsanity.com/paper/1910.07954/) (2019)
- [LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis](https://mlsanity.com/paper/2103.15348/) (2021)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)

## Built on by (most cited first, 10 of 10)

- [American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers](https://mlsanity.com/paper/2308.12477/) (2023)
- [EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge](https://mlsanity.com/paper/2310.10050/) (2023)
- [Newswire: A Large-Scale Structured Database of a Century of Historical News](https://mlsanity.com/paper/2406.09490/) (2024)
- [Deep Learning for Economists](https://mlsanity.com/paper/2407.15339/) (2024)
- [Reading the unreadable: Creating a dataset of 19th century English newspapers using image-to-text language models](https://mlsanity.com/paper/2502.14901/) (2025)
- [DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms](https://mlsanity.com/paper/2503.03644/) (2025)
- [Quantifying Character Similarity with Vision Transformers](https://mlsanity.com/paper/2305.14672/) (2023)
- [Linking Representations with Multimodal Contrastive Learning](https://mlsanity.com/paper/2304.03464/) (2023)
- [Layout-Aware OCR for Black Digital Archives with Unsupervised Evaluation](https://mlsanity.com/paper/2509.13236/) (2025)
- [Chronos: The AI Co-Historian](https://mlsanity.com/paper/2604.03553/) (2026)

## Abstract (arXiv)

Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) - which jointly learns a vision and language model - is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters' visual features, it is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, the model opens new avenues for community engagement in making digital history more representative of documentary history.