# Real-time End-to-End Video Text Spotter with Contrastive Representation Learning

arXiv: https://arxiv.org/abs/2207.08417 · HTML: https://mlsanity.com/paper/2207.08417/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [EAST: An Efficient and Accurate Scene Text Detector](https://mlsanity.com/paper/1704.03155/) (2017)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [Robust Scene Text Recognition with Automatic Rectification](https://mlsanity.com/paper/1603.03915/) (2016)
- [Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network](https://mlsanity.com/paper/1908.05900/) (2019)
- [Detecting Text in Natural Image with Connectionist Text Proposal Network](https://mlsanity.com/paper/1609.03605/) (2016)
- [PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text](https://mlsanity.com/paper/2105.00405/) (2021)
- [Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection](https://mlsanity.com/paper/2203.15221/) (2022)
- [A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer](https://mlsanity.com/paper/2112.04888/) (2021)
- [Semi-TCL: Semi-Supervised Track Contrastive Representation Learning](https://mlsanity.com/paper/2107.02396/) (2021)
- [End-to-End Video Text Spotting with Transformer](https://mlsanity.com/paper/2203.10539/) (2022)
- [You Only Recognize Once: Towards Fast Video Text Spotting](https://mlsanity.com/paper/1903.03299/) (2019)
- [Synthetic-to-Real Unsupervised Domain Adaptation for Scene Text Detection in the Wild](https://mlsanity.com/paper/2009.01766/) (2020)
- [TextCohesion: Detecting Text for Arbitrary Shapes](https://mlsanity.com/paper/1904.12640/) (2019)
- [Contrastive Learning of Semantic and Visual Representations for Text Tracking](https://mlsanity.com/paper/2112.14976/) (2021)

## Built on by (most cited first, 7 of 7)

- [A Large Cross-Modal Video Retrieval Dataset with Reading Comprehension](https://mlsanity.com/paper/2305.03347/) (2023)
- [Scalable Mask Annotation for Video Text Spotting](https://mlsanity.com/paper/2305.01443/) (2023)
- [GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching](https://mlsanity.com/paper/2401.07080/) (2024)
- [FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimation](https://mlsanity.com/paper/2305.03327/) (2023)
- [ICDAR 2023 Video Text Reading Competition for Dense and Small Text](https://mlsanity.com/paper/2304.04376/) (2023)
- [GloTSFormer: Global Video Text Spotting Transformer](https://mlsanity.com/paper/2401.03694/) (2024)
- [LOGO: Video Text Spotting with Language Collaboration and Glyph Perception Model](https://mlsanity.com/paper/2405.19194/) (2024)

## Abstract (arXiv)

Video text spotting(VTS) is the task that requires simultaneously detecting, tracking and recognizing text in the video. Existing video text spotting methods typically develop sophisticated pipelines and multiple models, which is not friend for real-time applications. Here we propose a real-time end-to-end video text spotter with Contrastive Representation learning (CoText). Our contributions are three-fold: 1) CoText simultaneously address the three tasks (e.g., text detection, tracking, recognition) in a real-time end-to-end trainable framework. 2) With contrastive learning, CoText models long-range dependencies and learning temporal information across multiple frames. 3) A simple, lightweight architecture is designed for effective and accurate performance, including GPU-parallel detection post-processing, CTC-based recognition head with Masked RoI. Extensive experiments show the superiority of our method. Especially, CoText achieves an video text spotting IDF1 of 72.0% at 41.0 FPS on ICDAR2015video, with 10.5% and 32.0 FPS improvement the previous best method. The code can be found at github.com/weijiawu/CoText.