# FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimation

arXiv: https://arxiv.org/abs/2305.03327 · HTML: https://mlsanity.com/paper/2305.03327/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 20 of 20)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models](https://mlsanity.com/paper/2303.11681/) (2023)
- [EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow](https://mlsanity.com/paper/1501.02565/) (2015)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)
- [Verisimilar Image Synthesis for Accurate Detection and Recognition of Texts in Scenes](https://mlsanity.com/paper/1807.03021/) (2018)
- [A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer](https://mlsanity.com/paper/2112.04888/) (2021)
- [End-to-End Video Text Spotting with Transformer](https://mlsanity.com/paper/2203.10539/) (2022)
- [You Only Recognize Once: Towards Fast Video Text Spotting](https://mlsanity.com/paper/1903.03299/) (2019)
- [Synthetic-to-Real Unsupervised Domain Adaptation for Scene Text Detection in the Wild](https://mlsanity.com/paper/2009.01766/) (2020)
- [ICDAR 2023 Video Text Reading Competition for Dense and Small Text](https://mlsanity.com/paper/2304.04376/) (2023)
- [Real-time End-to-End Video Text Spotter with Contrastive Representation Learning](https://mlsanity.com/paper/2207.08417/) (2022)
- [TextCohesion: Detecting Text for Arbitrary Shapes](https://mlsanity.com/paper/1904.12640/) (2019)

## Built on by (most cited first, 10 of 10)

- [DSText V2: A Comprehensive Video Text Spotting Dataset for Dense and Small Text](https://mlsanity.com/paper/2312.01938/) (2023)
- [End-to-end Remote Sensing Change Detection of Unregistered Bi-temporal Images for Natural Disasters](https://mlsanity.com/paper/2307.15128/) (2023)
- [Edge-guided and Class-balanced Active Learning for Semantic Segmentation of Aerial Images](https://mlsanity.com/paper/2405.18078/) (2024)
- [The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attention](https://mlsanity.com/paper/2405.17776/) (2024)
- [Lifelong Learning and Selective Forgetting via Contrastive Strategy](https://mlsanity.com/paper/2405.18663/) (2024)
- [Organizing Background to Explore Latent Classes for Incremental Few-shot Semantic Segmentation](https://mlsanity.com/paper/2405.19568/) (2024)
- [Adaptive Segmentation Network for Scene Text Detection](https://mlsanity.com/paper/2307.15029/) (2023)
- [VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization](https://mlsanity.com/paper/2404.19652/) (2024)
- [MagicSeg: Open-World Segmentation Pretraining via Counterfactural Diffusion-Based Auto-Generation](https://mlsanity.com/paper/2603.19575/) (2026)
- [CrossWeaver: Cross-modal Weaving for Arbitrary-Modality Semantic Segmentation](https://mlsanity.com/paper/2604.02948/) (2026)

## Abstract (arXiv)

Current video text spotting methods can achieve preferable performance, powered with sufficient labeled training data. However, labeling data manually is time-consuming and labor-intensive. To overcome this, using low-cost synthetic data is a promising alternative. This paper introduces a novel video text synthesis technique called FlowText, which utilizes optical flow estimation to synthesize a large amount of text video data at a low cost for training robust video text spotters. Unlike existing methods that focus on image-level synthesis, FlowText concentrates on synthesizing temporal information of text instances across consecutive frames using optical flow. This temporal information is crucial for accurately tracking and spotting text in video sequences, including text movement, distortion, appearance, disappearance, shelter, and blur. Experiments show that combining general detectors like TransDETR with the proposed FlowText produces remarkable results on various datasets, such as ICDAR2015video and ICDAR2013video. Code is available at https://github.com/callsys/FlowText.