mlsanityData to 13 Oct 2026

Can Scene Text Recognition Read Rare Compositions?

arXiv 2609.00816 ↗First posted 1 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Scene text recognition is reported as 89--97% accurate on the six standard benchmarks, and the problem is widely treated as saturated. We present an alternative reading. When the same test images are stratified jointly by ground-truth word rarity and character n-gram novelty against a reference corpus, accuracy at the rare-word x rare-trigram corner of the resulting 5x5 grid drops 10--18 pt below the q3/q3 centre across nine English specialised recognisers, and the same direction (corner below centre) holds on all 13 of 13 (language, model) pairs we test across four writing systems (Latin, Han, Han+kana, Arabic). The drop is not a capacity bottleneck. A 6x vision-backbone scale-up (CLIP4STR-Base 158M -> CLIP4STR-Huge 1.0B, OpenCLIP ViT-H/14 LAION-2B) leads every benchmark in aggregate accuracy yet leaves the stress corner unchanged (86.9 -> 86.5, within paired-bootstrap noise). Four converging probes--layer-wise probing, confidence-when-wrong, attention re-balancing, and a cross-script commit-vs-abstain error split--localise the failure to the autoregressive decoder's lexical prior. We then ask how much of the gap existing techniques recover. Of 16 non-architectural mitigations, the largest mean q5/q5 gain is +1.3 pt and none clears the paired-bootstrap noise floor; the only intervention that does is the architectural shift from autoregressive to CTC decoding (SVTRv2, +2.5 pt, p=0.02, n=474). A confidence-routed AR-CTC ensemble adds a directionally consistent +0.6 pt that stays within noise, and its dominant learned coefficient is each model's own minimum-softmax confidence--independently echoing the mechanism above. No configuration we test improves both the compositional corner and aggregate accuracy. The rare-input long tail thus points to architectural change rather than added capacity.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

22 of 33 references.

References · 22

  1. 2017On Calibration of Modern Neural Networkscited by 4,902
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,510
  3. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,450
  4. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  5. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 606
  6. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  7. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 462
  8. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 261
  9. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 239
  10. 2014Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognitioncited by 214
  11. 2021Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognitioncited by 194
  12. 2019What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysiscited by 191
10 more
  1. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 177
  2. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  3. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 148
  4. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  5. 2021Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Studycited by 53
  6. 2023Revisiting Scene Text Recognition: A Data Perspectivecited by 45
  7. 2020Generalized Entropy Regularization or: There's Nothing Special about Label Smoothingcited by 40
  8. 2020On Vocabulary Reliance in Scene Text Recognitioncited by 38
  9. 2022What Knowledge Gets Distilled in Knowledge Distillation?cited by 26
  10. 2020Maximum Entropy Regularization and Chinese Text Recognitioncited by 4

Built on by

Not cited yet.