# Sequence-to-Sequence Contrastive Learning for Text Recognition

arXiv: https://arxiv.org/abs/2012.10873 · HTML: https://mlsanity.com/paper/2012.10873/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2020.

## Built on by (most cited first, 30 of 70)

- [LaTr: Layout-Aware Transformer for Scene-Text VQA](https://mlsanity.com/paper/2112.12494/) (2021)
- [Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution](https://mlsanity.com/paper/2305.07498/) (2023)
- [Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo](https://mlsanity.com/paper/2203.17248/) (2022)
- [Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognition](https://mlsanity.com/paper/2207.00193/) (2022)
- [Revisiting Scene Text Recognition: A Data Perspective](https://mlsanity.com/paper/2307.08723/) (2023)
- [Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure](https://mlsanity.com/paper/2211.10738/) (2022)
- [OCR-IDL: OCR Annotations for Industry Document Library Dataset](https://mlsanity.com/paper/2202.12985/) (2022)
- [MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining](https://mlsanity.com/paper/2206.00311/) (2022)
- [DocVLM: Make Your VLM an Efficient Reader](https://mlsanity.com/paper/2412.08746/) (2024)
- [Self-supervised Character-to-Character Distillation for Text Recognition](https://mlsanity.com/paper/2211.00288/) (2022)
- [Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization](https://mlsanity.com/paper/2305.11965/) (2023)
- [GRAM: Global Reasoning for Multi-Page VQA](https://mlsanity.com/paper/2401.03411/) (2024)
- [SimAN: Exploring Self-Supervised Representation Learning of Scene Text via Similarity-Aware Normalization](https://mlsanity.com/paper/2203.10492/) (2022)
- [Instruction-Guided Scene Text Recognition](https://mlsanity.com/paper/2401.17851/) (2024)
- [Text-DIAE: A Self-Supervised Degradation Invariant Autoencoders for Text Recognition and Document Enhancement](https://mlsanity.com/paper/2203.04814/) (2022)
- [Multimodal Semi-Supervised Learning for Text Recognition](https://mlsanity.com/paper/2205.03873/) (2022)
- [CLIPTER: Looking at the Bigger Picture in Scene Text Recognition](https://mlsanity.com/paper/2301.07464/) (2023)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Hard Negative Sampling Strategies for Contrastive Representation Learning](https://mlsanity.com/paper/2206.01197/) (2022)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [TextAdaIN: Paying Attention to Shortcut Learning in Text Recognizers](https://mlsanity.com/paper/2105.03906/) (2021)
- [Out-of-Vocabulary Challenge Report](https://mlsanity.com/paper/2209.06717/) (2022)
- [SGBANet: Semantic GAN and Balanced Attention Network for Arbitrarily Oriented Scene Text Recognition](https://mlsanity.com/paper/2207.10256/) (2022)
- [Efficient OCR for Building a Diverse Digital History](https://mlsanity.com/paper/2304.02737/) (2023)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Mini-Batch Optimization of Contrastive Loss](https://mlsanity.com/paper/2307.05906/) (2023)
- [Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach](https://mlsanity.com/paper/2308.08806/) (2023)
- [Self-Supervised Pre-training with Symmetric Superimposition Modeling for Scene Text Recognition](https://mlsanity.com/paper/2405.05841/) (2024)
- [Evaluating Synthetic Pre-Training for Handwriting Processing Tasks](https://mlsanity.com/paper/2304.01842/) (2023)

## Abstract (arXiv)

We propose a framework for sequence-to-sequence contrastive learning (SeqCLR) of visual representations, which we apply to text recognition. To account for the sequence-to-sequence structure, each feature map is divided into different instances over which the contrastive loss is computed. This operation enables us to contrast in a sub-word level, where from each image we extract several positive pairs and multiple negative examples. To yield effective visual representations for text recognition, we further suggest novel augmentation heuristics, different encoder architectures and custom projection heads. Experiments on handwritten text and on scene text show that when a text decoder is trained on the learned representations, our method outperforms non-sequential contrastive methods. In addition, when the amount of supervision is reduced, SeqCLR significantly improves performance compared with supervised training, and when fine-tuned with 100% of the labels, our method achieves state-of-the-art results on standard handwritten text recognition benchmarks.