# Masked Next-Scale Prediction for Self-supervised Scene Text Recognition

arXiv: https://arxiv.org/abs/2605.14885 · HTML: https://mlsanity.com/paper/2605.14885/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [HART: Efficient Visual Generation with Hybrid Autoregressive Transformer](https://mlsanity.com/paper/2410.10812/) (2024)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learning](https://mlsanity.com/paper/2006.11476/) (2020)
- [From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network](https://mlsanity.com/paper/2108.09661/) (2021)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [Video Cloze Procedure for Self-Supervised Spatio-Temporal Learning](https://mlsanity.com/paper/2001.00294/) (2020)
- [Sequence-to-Sequence Contrastive Learning for Text Recognition](https://mlsanity.com/paper/2012.10873/) (2020)
- [Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognition](https://mlsanity.com/paper/2207.00193/) (2022)
- [Revisiting Scene Text Recognition: A Data Perspective](https://mlsanity.com/paper/2307.08723/) (2023)
- [PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition](https://mlsanity.com/paper/2109.04145/) (2021)
- [DTrOCR: Decoder-only Transformer for Optical Character Recognition](https://mlsanity.com/paper/2308.15996/) (2023)
- [Visual Autoregressive Modeling for Image Super-Resolution](https://mlsanity.com/paper/2501.18993/) (2025)
- [Multi-modal Text Recognition Networks: Interactive Enhancements between Visual and Semantic Features](https://mlsanity.com/paper/2111.15263/) (2021)
- [Self-supervised Character-to-Character Distillation for Text Recognition](https://mlsanity.com/paper/2211.00288/) (2022)
- [Dense Semantic Contrast for Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2109.07756/) (2021)
- [SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition](https://mlsanity.com/paper/2411.15858/) (2024)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition](https://mlsanity.com/paper/2310.04999/) (2023)
- [IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognition](https://mlsanity.com/paper/2312.11923/) (2023)
- [TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition](https://mlsanity.com/paper/2305.05322/) (2023)

## Abstract (arXiv)

Scene Text Recognition requires modeling visual structures that evolve from coarse layouts to fine-grained character strokes. Training such models relies on large amounts of annotated data. Recent self-supervised approaches, such as Masked Image Modeling (MIM), alleviate this dependency by leveraging large-scale unlabeled data. Yet most existing MIM methods operate at a single spatial scale and fail to capture the hierarchical nature of scene text. In this work, we introduce Masked Next-Scale Prediction (MNSP), a unified self-supervised framework designed to explicitly model cross-scale structural evolution. The framework incorporates Next-Scale Prediction (NSP), which learns hierarchical representations by predicting higher-resolution features from lower-resolution contexts. Naive scale prediction, however, tends to produce spatially diffuse attention, directing the model toward background regions rather than textual structures. MNSP resolves this limitation by jointly learning cross-scale prediction and masked image reconstruction. NSP captures global layout priors across resolutions, while masked reconstruction imposes strong local constraints that guide attention toward informative text regions. A Multi-scale Linguistic Alignment module further maintains semantic consistency across different resolutions. Extensive experiments demonstrate that MNSP achieves state-of-the-art performance, reaching 86.2% average accuracy on the challenging Union14M benchmark and 96.7% across six standard datasets. Additional analyses show that our method improves robustness under extreme scale and layout variations. Code is available at https://github.com/CzhczhcHczh/MNSP