# Decoupling Visual-Semantic Feature Learning for Robust Scene Text Recognition

arXiv: https://arxiv.org/abs/2111.12351 · HTML: https://mlsanity.com/paper/2111.12351/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Reading Text in the Wild with Convolutional Neural Networks](https://mlsanity.com/paper/1412.1842/) (2014)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Focusing Attention: Towards Accurate Text Recognition in Natural Images](https://mlsanity.com/paper/1709.02054/) (2017)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [ESIR: End-to-end Scene Text Recognition via Iterative Image Rectification](https://mlsanity.com/paper/1812.05824/) (2018)
- [Recursive Recurrent Nets with Attention Modeling for OCR in the Wild](https://mlsanity.com/paper/1603.03101/) (2016)
- [TextScanner: Reading Characters in Order for Robust Scene Text Recognition](https://mlsanity.com/paper/1912.12422/) (2019)
- [Scene Text Recognition from Two-Dimensional Perspective](https://mlsanity.com/paper/1809.06508/) (2018)
- [GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition](https://mlsanity.com/paper/2002.01276/) (2020)
- [Symmetry-constrained Rectification Network for Scene Text Recognition](https://mlsanity.com/paper/1908.01957/) (2019)
- [Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM](https://mlsanity.com/paper/1706.02737/) (2017)
- [On Vocabulary Reliance in Scene Text Recognition](https://mlsanity.com/paper/2005.03959/) (2020)
- [Aggregation Cross-Entropy for Sequence Recognition](https://mlsanity.com/paper/1904.08364/) (2019)
- [Adaptive Embedding Gate for Attention-Based Scene Text Recognition](https://mlsanity.com/paper/1908.09475/) (2019)

## Built on by (most cited first, 2 of 2)

- [LISTER: Neighbor Decoding for Length-Insensitive Scene Text Recognition](https://mlsanity.com/paper/2308.12774/) (2023)
- [A3S: Adversarial learning of semantic representations for Scene-Text Spotting](https://mlsanity.com/paper/2302.10641/) (2023)

## Abstract (arXiv)

Semantic information has been proved effective in scene text recognition. Most existing methods tend to couple both visual and semantic information in an attention-based decoder. As a result, the learning of semantic features is prone to have a bias on the limited vocabulary of the training set, which is called vocabulary reliance. In this paper, we propose a novel Visual-Semantic Decoupling Network (VSDN) to address the problem. Our VSDN contains a Visual Decoder (VD) and a Semantic Decoder (SD) to learn purer visual and semantic feature representation respectively. Besides, a Semantic Encoder (SE) is designed to match SD, which can be pre-trained together by additional inexpensive large vocabulary via a simple word correction task. Thus the semantic feature is more unbiased and precise to guide the visual feature alignment and enrich the final character representation. Experiments show that our method achieves state-of-the-art or competitive results on the standard benchmarks, and outperforms the popular baseline by a large margin under circumstances where the training set has a small size of vocabulary.