mlsanityData to 14 Oct 2026

Scene Text Recognition with Permuted Autoregressive Sequence Models

arXiv 2207.06966 ↗First posted 14 Jul 2022cs.CV cs.CLcited by 149

Not on the map. Why?

Abstract (arXiv)

Context-aware STR methods typically use internal autoregressive (AR) language models (LM). Inherent limitations of AR models motivated two-stage methods which employ an external LM. The conditional independence of the external LM on the input image may cause it to erroneously rectify correct predictions, leading to significant inefficiencies. Our method, PARSeq, learns an ensemble of internal AR LMs with shared weights using Permutation Language Modeling. It unifies context-free non-AR and context-aware AR inference, and iterative refinement using bidirectional context. Using synthetic training data, PARSeq achieves state-of-the-art (SOTA) results in STR benchmarks (91.9% accuracy) and more challenging datasets. It establishes new SOTA results (96.0% accuracy) when trained on real data. PARSeq is optimal on accuracy vs parameter count, FLOPS, and latency because of its simple, unified structure and parallel token processing. Due to its extensive use of attention, it is robust on arbitrarily-oriented text which is common in real-world images. Code, pretrained weights, and data are available at: https://github.com/baudm/parseq.

Key facts. Cited by 149 papers in the map (data through 14 Oct 2026). Builds on 59 of its 84 references that are in the map. First posted on arXiv 14 Jul 2022.

Builds on

59 of 84 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  12. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
38 more
  1. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  2. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  3. 2021Early Convolutions Help Transformers See Bettercited by 396
  4. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  5. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  6. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 236
  7. 2018Adaptive Input Representations for Neural Language Modelingcited by 226
  8. 2014Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognitioncited by 214
  9. 2021Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognitioncited by 194
  10. 2019What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysiscited by 191
  11. 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 171
  12. 2019ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)cited by 147
  13. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 139
  14. 2020Towards Accurate Scene Text Recognition with Semantic Reasoning Networkscited by 131
  15. 2019ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboardcited by 128
  16. 2019ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVTcited by 122
  17. 2016Robust Scene Text Recognition with Automatic Rectificationcited by 121
  18. 2020SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognitioncited by 119
  19. 2018Scene Text Detection and Recognition: The Deep Learning Eracited by 114
  20. 2017Focusing Attention: Towards Accurate Text Recognition in Natural Imagescited by 110
  21. 2017ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)cited by 105
  22. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 105
  23. 2021From Two to One: A New Scene Text Recognizer with Visual Language Modeling Networkcited by 97
  24. 2020RobustScanner: Dynamically Enhancing Positional Clues for Robust Text Recognitioncited by 85
  25. 2021Vision Transformer for Fast and Efficient Scene Text Recognitioncited by 81
  26. 2016Recursive Recurrent Nets with Attention Modeling for OCR in the Wildcited by 74
  27. 2018NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognitioncited by 73
  28. 2019Rosetta: Large scale system for text detection and recognition in imagescited by 69
  29. 2020SCATTER: Selective Context Attentional Scene Text Recognizercited by 69
  30. 2013A Deep and Tractable Density Estimatorcited by 68
  31. 2019TextScanner: Reading Characters in Order for Robust Scene Text Recognitioncited by 68
  32. 2021What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labelscited by 63
  33. 2019On Recognizing Texts of Arbitrary Shapes with 2D Self-Attentioncited by 59
  34. 2021Fast and Accurate Model Scalingcited by 46
  35. 2021Open Images V5 Text Annotation and Yet Another Mask Text Spottercited by 46
  36. 2020UnrealText: Synthesizing Realistic Scene Text Images from the Unreal Worldcited by 43
  37. 2020BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretrainingcited by 33
  38. 2020Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Ordercited by 28

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 43; 2024: 43; 2025: 28; 2026: 33.

149 papers.

Sort:

By area:

  1. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  2. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  3. 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
  4. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  5. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  6. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  7. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
  8. 2023UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Modelscited by 59
  9. 2023Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizercited by 53
  10. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 53
  11. 2023Revisiting Scene Text Recognition: A Data Perspectivecited by 45
  12. 2023DTrOCR: Decoder-only Transformer for Optical Character Recognitioncited by 40
38 more
  1. 2023FuseCap: Leveraging Large Language Models for Enriched Fused Image Captionscited by 33
  2. 2023CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Modelcited by 30
  3. 2022Self-supervised Character-to-Character Distillation for Text Recognitioncited by 27
  4. 2024CoLLaVO: Crayon Large Language and Vision mOdelcited by 26
  5. 2024SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognitioncited by 24
  6. 2023ICDAR 2023 Competition on Hierarchical Text Detection and Recognitioncited by 23
  7. 2023Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognitioncited by 20
  8. 2023Context Perception Parallel Decoder for Scene Text Recognitioncited by 19
  9. 2024DreamText: High Fidelity Scene Text Synthesiscited by 18
  10. 2024Instruction-Guided Scene Text Recognitioncited by 17
  11. 2023CLIPTER: Looking at the Bigger Picture in Scene Text Recognitioncited by 16
  12. 2023Scene Text Image Super-resolution based on Text-conditional Diffusion Modelscited by 15
  13. 2023Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognitioncited by 14
  14. 2024Layout Agnostic Scene Text Image Synthesis with Diffusion Modelscited by 14
  15. 2023LISTER: Neighbor Decoding for Length-Insensitive Scene Text Recognitioncited by 13
  16. 2025TokBench: Evaluating Your Visual Tokenizer before Visual Generationcited by 13
  17. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  18. 2023IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognitioncited by 12
  19. 2024GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matchingcited by 12
  20. 2024UNIT: Unifying Image and Text Recognition in One Vision Encodercited by 12
  21. 2024A General Framework for Jersey Number Recognition in Sports Videocited by 11
  22. 2024Out of Length Text Recognition with Sub-String Matchingcited by 11
  23. 2025PICD: Versatile Perceptual Image Compression with Diffusion Renderingcited by 11
  24. 2024Class-Aware Mask-Guided Feature Refinement for Scene Text Recognitioncited by 10
  25. 2023ICDAR 2023 Video Text Reading Competition for Dense and Small Textcited by 9
  26. 2023TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognitioncited by 9
  27. 2024Bridging the Gap Between End-to-End and Two-Step Text Spottingcited by 9
  28. 2023Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approachcited by 8
  29. 2023Hierarchical Text Spotter for Joint Text Spotting and Layout Analysiscited by 8
  30. 2024Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processingcited by 8
  31. 2024Self-Supervised Pre-training with Symmetric Superimposition Modeling for Scene Text Recognitioncited by 8
  32. 2024AnyTrans: Translate AnyText in the Image with Large Scale Modelscited by 8
  33. 2024DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthinesscited by 8
  34. 2024TextSSR: Diffusion-based Data Synthesis for Scene Text Recognitioncited by 8
  35. 2023DiffusionSTR: Diffusion Model for Scene Text Recognitioncited by 7
  36. 2023Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognitioncited by 7
  37. 2023An Empirical Study of Scaling Law for OCRcited by 7
  38. 2025PRIM: Towards Practical In-Image Multilingual Machine Translationcited by 7

50 most cited of 149 All 149 (JSON)