# Extending TrOCR for Text Localization-Free OCR of Full-Page Scanned Receipt Images

arXiv: https://arxiv.org/abs/2212.05525 · HTML: https://mlsanity.com/paper/2212.05525/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 11 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network](https://mlsanity.com/paper/1908.05900/) (2019)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [An end-to-end TextSpotter with Explicit Alignment and Attention](https://mlsanity.com/paper/1803.03474/) (2018)
- [Transformer-based HTR for Historical Documents](https://mlsanity.com/paper/2203.11008/) (2022)
- [Visual-Semantic Transformer for Scene Text Recognition](https://mlsanity.com/paper/2112.00948/) (2021)
- [End-to-End Text Recognition with Hybrid HMM Maxout Models](https://mlsanity.com/paper/1310.1811/) (2013)
- [Utilizing Resource-Rich Language Datasets for End-to-End Scene Text Recognition in Resource-Poor Languages](https://mlsanity.com/paper/2111.12276/) (2021)

## Built on by (most cited first, 5 of 5)

- [Digitization of Document and Information Extraction using OCR](https://mlsanity.com/paper/2506.11156/) (2025)
- [DOTA: Deformable Optimized Transformer Architecture for End-to-End Text Recognition with Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.04175/) (2025)
- [Benchmarking OCR Pipelines with Adaptive Enhancement for Multi-Domain Retail Bill Digitization](https://mlsanity.com/paper/2604.25176/) (2026)
- [TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents](https://mlsanity.com/paper/2608.07917/) (2026)
- [Systematic Literature Review of Machine Learning Models and Applications for Text Recognition](https://mlsanity.com/paper/2608.26500/) (2026)

## Abstract (arXiv)

Digitization of scanned receipts aims to extract text from receipt images and save it into structured documents. This is usually split into two sub-tasks: text localization and optical character recognition (OCR). Most existing OCR models only focus on the cropped text instance images, which require the bounding box information provided by a text region detection model. Introducing an additional detector to identify the text instance images in advance adds complexity, however instance-level OCR models have very low accuracy when processing the whole image for the document-level OCR, such as receipt images containing multiple text lines arranged in various layouts. To this end, we propose a localization-free document-level OCR model for transcribing all the characters in a receipt image into an ordered sequence end-to-end. Specifically, we finetune the pretrained instance-level model TrOCR with randomly cropped image chunks, and gradually increase the image chunk size to generalize the recognition ability from instance images to full-page images. In our experiments on the SROIE receipt OCR dataset, the model finetuned with our strategy achieved 64.4 F1-score and a 22.8% character error rate (CER), respectively, which outperforms the baseline results with 48.5 F1-score and 50.6% CER. The best model, which splits the full image into 15 equally sized chunks, gives 87.8 F1-score and 4.98% CER with minimal additional pre or post-processing of the output. Moreover, the characters in the generated document-level sequences are arranged in the reading order, which is practical for real-world applications.