# An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

arXiv: https://arxiv.org/abs/1507.05717 · HTML: https://mlsanity.com/paper/1507.05717/

## Key facts

- Cited by 520 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2015.

## Built on by (most cited first, 30 of 520)

- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024)
- [TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text](https://mlsanity.com/paper/2105.05486/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System](https://mlsanity.com/paper/2206.03001/) (2022)
- [DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding](https://mlsanity.com/paper/2311.11810/) (2023)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding](https://mlsanity.com/paper/2404.05225/) (2024)
- [StrucTexT: Structured Text Understanding with Multi-Modal Transformers](https://mlsanity.com/paper/2108.02923/) (2021)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting](https://mlsanity.com/paper/2105.03620/) (2021)
- [TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrieval](https://mlsanity.com/paper/2104.08271/) (2021)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition](https://mlsanity.com/paper/2403.19128/) (2024)
- [SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition](https://mlsanity.com/paper/2203.10209/) (2022)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [Syntax-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2203.01601/) (2022)
- [SPTS v2: Single-Point Scene Text Spotting](https://mlsanity.com/paper/2301.01635/) (2023)
- [PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text](https://mlsanity.com/paper/2105.00405/) (2021)
- [What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels](https://mlsanity.com/paper/2103.04400/) (2021)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)

## Abstract (arXiv)

Image-based sequence recognition has been a long-standing research topic in computer vision. In this paper, we investigate the problem of scene text recognition, which is among the most important and challenging tasks in image-based sequence recognition. A novel neural network architecture, which integrates feature extraction, sequence modeling and transcription into a unified framework, is proposed. Compared with previous systems for scene text recognition, the proposed architecture possesses four distinctive properties: (1) It is end-to-end trainable, in contrast to most of the existing algorithms whose components are separately trained and tuned. (2) It naturally handles sequences in arbitrary lengths, involving no character segmentation or horizontal scale normalization. (3) It is not confined to any predefined lexicon and achieves remarkable performances in both lexicon-free and lexicon-based scene text recognition tasks. (4) It generates an effective yet much smaller model, which is more practical for real-world application scenarios. The experiments on standard benchmarks, including the IIIT-5K, Street View Text and ICDAR datasets, demonstrate the superiority of the proposed algorithm over the prior arts. Moreover, the proposed algorithm performs well in the task of image-based music score recognition, which evidently verifies the generality of it.