# DTrOCR: Decoder-only Transformer for Optical Character Recognition

arXiv: https://arxiv.org/abs/2308.15996 · HTML: https://mlsanity.com/paper/2308.15996/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 68 references that are in the map.
- First posted on arXiv 30 Aug 2023.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction](https://mlsanity.com/paper/2103.10213/) (2021)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard](https://mlsanity.com/paper/1912.09641/) (2019)
- [Reading Text in the Wild with Convolutional Neural Networks](https://mlsanity.com/paper/1412.1842/) (2014)
- [ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT](https://mlsanity.com/paper/1909.07741/) (2019)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)](https://mlsanity.com/paper/1708.09585/) (2017)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)

## Built on by (most cited first, 30 of 40)

- [TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model](https://mlsanity.com/paper/2503.18278/) (2025)
- [DocVLM: Make Your VLM an Efficient Reader](https://mlsanity.com/paper/2412.08746/) (2024)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [HATFormer: Historic Handwritten Arabic Text Recognition with Transformers](https://mlsanity.com/paper/2410.02179/) (2024)
- [DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness](https://mlsanity.com/paper/2412.00151/) (2024)
- [TextBlockV2: Towards Precise-Detection-Free Scene Text Spotting with Pre-trained Language Model](https://mlsanity.com/paper/2403.10047/) (2024)
- [RL-LOGO: Deep Reinforcement Learning Localization for Logo Recognition](https://mlsanity.com/paper/2312.16792/) (2023)
- [Mixed Text Recognition with Efficient Parameter Fine-Tuning and Transformer](https://mlsanity.com/paper/2404.12734/) (2024)
- [General Detection-based Text Line Recognition](https://mlsanity.com/paper/2409.17095/) (2024)
- [TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques](https://mlsanity.com/paper/2312.02125/) (2023)
- [InkFM: A Foundational Model for Full-Page Online Handwritten Note Understanding](https://mlsanity.com/paper/2503.23081/) (2025)
- [MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning](https://mlsanity.com/paper/2505.20513/) (2025)
- [JSTR: Judgment Improves Scene Text Recognition](https://mlsanity.com/paper/2404.05967/) (2024)
- [Spatial Context-based Self-Supervised Learning for Handwritten Text Recognition](https://mlsanity.com/paper/2404.11585/) (2024)
- [JaPOC: Japanese Post-OCR Correction Benchmark using Vouchers](https://mlsanity.com/paper/2409.19948/) (2024)
- [Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription](https://mlsanity.com/paper/2502.20295/) (2025)
- [Practical Fine-Tuning of Autoregressive Models on Limited Handwritten Texts](https://mlsanity.com/paper/2503.19546/) (2025)
- [Leveraging Contrast Information for Efficient Document Shadow Removal](https://mlsanity.com/paper/2504.00385/) (2025)
- [HTR-ConvText: Leveraging Convolution and Textual Information for Handwritten Text Recognition](https://mlsanity.com/paper/2512.05021/) (2025)
- [DRetHTR: Linear-Time Decoder-Only Retentive Network for Handwritten Text Recognition](https://mlsanity.com/paper/2602.17387/) (2026)
- [Few-shot Writer Adaptation via Multimodal In-Context Learning](https://mlsanity.com/paper/2603.29450/) (2026)
- [Democratizing the medieval English legal tradition](https://mlsanity.com/paper/2605.00977/) (2026)
- [EpiSAM: Character Segmentation in Challenging Stone Inscriptions](https://mlsanity.com/paper/2606.28859/) (2026)
- [Segmentation-free Connectionist Temporal Classification loss based OCR Model for Text Captcha Classification](https://mlsanity.com/paper/2402.05417/) (2024)
- [Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval](https://mlsanity.com/paper/2501.14846/) (2025)
- [Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation](https://mlsanity.com/paper/2503.16184/) (2025)
- [Coarse-to-Fine Learning for Multi-Pipette Localisation in Robot-Assisted In Vivo Patch-Clamp](https://mlsanity.com/paper/2504.01044/) (2025)
- [A Lightweight Multi-Module Fusion Approach for Korean Character Recognition](https://mlsanity.com/paper/2504.05770/) (2025)
- [Preserving Privacy Without Compromising Accuracy: Machine Unlearning for Handwritten Text Recognition](https://mlsanity.com/paper/2504.08616/) (2025)

## Abstract (arXiv)

Typical text recognition methods rely on an encoder-decoder structure, in which the encoder extracts features from an image, and the decoder produces recognized text from these features. In this study, we propose a simpler and more effective method for text recognition, known as the Decoder-only Transformer for Optical Character Recognition (DTrOCR). This method uses a decoder-only Transformer to take advantage of a generative language model that is pre-trained on a large corpus. We examined whether a generative language model that has been successful in natural language processing can also be effective for text recognition in computer vision. Our experiments demonstrated that DTrOCR outperforms current state-of-the-art methods by a large margin in the recognition of printed, handwritten, and scene text in both English and Chinese.