mlsanityData to 7 Oct 2026

End-to-End Video Text Spotting with Transformer

arXiv 2203.10539 ↗First posted 20 Mar 2022cs.CV cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Recent video text spotting methods usually require the three-staged pipeline, i.e., detecting text in individual images, recognizing localized text, tracking text streams with post-processing to generate final results. These methods typically follow the tracking-by-match paradigm and develop sophisticated pipelines. In this paper, rooted in Transformer sequence modeling, we propose a simple, but effective end-to-end video text DEtection, Tracking, and Recognition framework (TransDETR). TransDETR mainly includes two advantages: 1) Different from the explicit match paradigm in the adjacent frame, TransDETR tracks and recognizes each text implicitly by the different query termed text query over long-range temporal sequence (more than 7 frames). 2) TransDETR is the first end-to-end trainable video text spotting framework, which simultaneously addresses the three sub-tasks (e.g., text detection, tracking, recognition). Extensive experiments in four video text datasets (i.e.,ICDAR2013 Video, ICDAR2015 Video, Minetto, and YouTube Video Text) are conducted to demonstrate that TransDETR achieves state-of-the-art performance with up to around 8.0% improvements on video text spotting tasks. The code of TransDETR can be found at https://github.com/weijiawu/TransDETR.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 41 of its 81 references that are in the map. First posted on arXiv 20 Mar 2022.

Builds on

41 of 81 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  12. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 519
29 more
  1. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  2. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  3. 2020Tracking Objects as Pointscited by 483
  4. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  5. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  6. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  7. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  8. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  9. 2019Character Region Awareness for Text Detectioncited by 273
  10. 2017EAST: An Efficient and Accurate Scene Text Detectorcited by 248
  11. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 235
  12. 2021Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Trackingcited by 220
  13. 2014Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognitioncited by 214
  14. 2017Arbitrary-Oriented Scene Text Detection via Rotation Proposalscited by 176
  15. 2019Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapescited by 171
  16. 2020ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Networkcited by 170
  17. 2019Shape Robust Text Detection with Progressive Scale Expansion Networkcited by 147
  18. 2018FOTS: Fast Oriented Text Spotting with a Unified Networkcited by 109
  19. 2021PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Textcited by 65
  20. 2017Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networkscited by 56
  21. 2018An end-to-end TextSpotter with Explicit Alignment and Attentioncited by 50
  22. 2021A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformercited by 26
  23. 2020RoadText-1K: Text Detection & Recognition Dataset for Driving Videoscited by 25
  24. 2019You Only Recognize Once: Towards Fast Video Text Spottingcited by 21
  25. 2019Towards End-to-End Text Spotting in Natural Scenescited by 19
  26. 2020Synthetic-to-Real Unsupervised Domain Adaptation for Scene Text Detection in the Wildcited by 15
  27. 2019TextCohesion: Detecting Text for Arbitrary Shapescited by 5
  28. 2020RRPN++: Guidance Towards More Accurate Scene Text Detectioncited by 4
  29. 2021Contrastive Learning of Semantic and Visual Representations for Text Trackingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 6; 2025: 2; 2026: 2.

21 papers.

Sort:

By area:

  1. 2022Tracking Objects as Pixel-wise Distributionscited by 36
  2. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  3. 2023A Large Cross-Modal Video Retrieval Dataset with Reading Comprehensioncited by 31
  4. 2024PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Samplingcited by 14
  5. 2023Scalable Mask Annotation for Video Text Spottingcited by 13
  6. 2023Continual Learning for Image Segmentation with Dynamic Querycited by 12
  7. 2024GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matchingcited by 12
  8. 2022Explore Faster Localization Learning For Scene Text Detectioncited by 10
  9. 2023FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimationcited by 10
  10. 2023ICDAR 2023 Video Text Reading Competition for Dense and Small Textcited by 9
  11. 2022Real-time End-to-End Video Text Spotter with Contrastive Representation Learningcited by 7
  12. 2023DSText V2: A Comprehensive Video Text Spotting Dataset for Dense and Small Textcited by 7
9 more
  1. 2023Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrievalcited by 4
  2. 2025GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarkingcited by 4
  3. 2025SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQAcited by 2
  4. 2023Video text tracking for dense and small text based on pp-yoloe-r and sort algorithmcited by 0
  5. 2024GloTSFormer: Global Video Text Spotting Transformercited by 0
  6. 2024VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalizationcited by 0
  7. 2024LOGO: Video Text Spotting with Language Collaboration and Glyph Perception Modelcited by 0
  8. 2026TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillancecited by 0
  9. 2026Back-Tracking from Clarity: Self-Learning to See Text from Afarcited by 0