# Masked and Permuted Implicit Context Learning for Scene Text Recognition

arXiv: https://arxiv.org/abs/2305.16172 · HTML: https://mlsanity.com/paper/2305.16172/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 58 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)](https://mlsanity.com/paper/1909.07145/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Reading Text in the Wild with Convolutional Neural Networks](https://mlsanity.com/paper/1412.1842/) (2014)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Scene Text Detection and Recognition: The Deep Learning Era](https://mlsanity.com/paper/1811.04256/) (2018)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [TextScanner: Reading Characters in Order for Robust Scene Text Recognition](https://mlsanity.com/paper/1912.12422/) (2019)
- [Scene Text Recognition from Two-Dimensional Perspective](https://mlsanity.com/paper/1809.06508/) (2018)
- [GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition](https://mlsanity.com/paper/2002.01276/) (2020)
- [PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition](https://mlsanity.com/paper/2109.04145/) (2021)
- [Multi-Granularity Prediction for Scene Text Recognition](https://mlsanity.com/paper/2209.03592/) (2022)
- [When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2207.11463/) (2022)
- [Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features](https://mlsanity.com/paper/2111.15263/) (2021)

## Built on by (most cited first, 4 of 4)

- [First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending](https://mlsanity.com/paper/2410.10168/) (2024)
- [IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognition](https://mlsanity.com/paper/2312.11923/) (2023)
- [Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2503.18746/) (2025)
- [Masked Next-Scale Prediction for Self-supervised Scene Text Recognition](https://mlsanity.com/paper/2605.14885/) (2026)

## Abstract (arXiv)

Scene Text Recognition (STR) is difficult because of the variations in text styles, shapes, and backgrounds. Though the integration of linguistic information enhances models' performance, existing methods based on either permuted language modeling (PLM) or masked language modeling (MLM) have their pitfalls. PLM's autoregressive decoding lacks foresight into subsequent characters, while MLM overlooks inter-character dependencies. Addressing these problems, we propose a masked and permuted implicit context learning network for STR, which unifies PLM and MLM within a single decoder, inheriting the advantages of both approaches. We utilize the training procedure of PLM, and to integrate MLM, we incorporate word length information into the decoding process and replace the undetermined characters with mask tokens. Besides, perturbation training is employed to train a more robust model against potential length prediction errors. Our empirical evaluations demonstrate the performance of our model. It not only achieves superior performance on the common benchmarks but also achieves a substantial improvement of 9.1% on the more challenging Union14M-Benchmark.