# Track the Answer: Extending TextVQA from Image to Video with Spatio-Temporal Clues

arXiv: https://arxiv.org/abs/2412.12502 · HTML: https://mlsanity.com/paper/2412.12502/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 44 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 3 of 3)

- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [Instruction-Guided Scene Text Recognition](https://mlsanity.com/paper/2401.17851/) (2024)

## Built on by (most cited first, 6 of 6)

- [VidText: Towards Comprehensive Evaluation for Video Text Understanding](https://mlsanity.com/paper/2505.22810/) (2025)
- [Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination](https://mlsanity.com/paper/2511.17490/) (2025)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQA](https://mlsanity.com/paper/2511.20190/) (2025)
- [VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA](https://mlsanity.com/paper/2605.04870/) (2026)
- [Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction](https://mlsanity.com/paper/2609.25769/) (2026)

## Abstract (arXiv)

Video text-based visual question answering (Video TextVQA) is a practical task that aims to answer questions by jointly reasoning textual and visual information in a given video. Inspired by the development of TextVQA in image domain, existing Video TextVQA approaches leverage a language model (e.g. T5) to process text-rich multiple frames and generate answers auto-regressively. Nevertheless, the spatio-temporal relationships among visual entities (including scene text and objects) will be disrupted and models are susceptible to interference from unrelated information, resulting in irrational reasoning and inaccurate answering. To tackle these challenges, we propose the TEA (stands for ``Track thE Answer'') method that better extends the generative TextVQA framework from image to video. TEA recovers the spatio-temporal relationships in a complementary way and incorporates OCR-aware clues to enhance the quality of reasoning questions. Extensive experiments on several public Video TextVQA datasets validate the effectiveness and generalization of our framework. TEA outperforms existing TextVQA methods, video-language pretraining methods and video large language models by great margins.