# Gather and Trace: Rethinking Video TextVQA from an Instance-oriented Perspective

arXiv: https://arxiv.org/abs/2508.04197 · HTML: https://mlsanity.com/paper/2508.04197/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 15 of 15)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [NVILA: Efficient Frontier Visual Language Models](https://mlsanity.com/paper/2412.04468/) (2024)
- [Video Question Answering: Datasets, Algorithms and Challenges](https://mlsanity.com/paper/2203.01225/) (2022)
- [Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding](https://mlsanity.com/paper/2501.07888/) (2025)
- [EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering](https://mlsanity.com/paper/2502.07411/) (2025)
- [PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance](https://mlsanity.com/paper/2411.02327/) (2024)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [A Simple Baseline for Knowledge-Based Visual Question Answering](https://mlsanity.com/paper/2310.13570/) (2023)
- [GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching](https://mlsanity.com/paper/2401.07080/) (2024)
- [VidText: Towards Comprehensive Evaluation for Video Text Understanding](https://mlsanity.com/paper/2505.22810/) (2025)
- [Video-Panda: Parameter-efficient Alignment for Encoder-free Video-Language Models](https://mlsanity.com/paper/2412.18609/) (2024)
- [EVE: Towards End-to-End Video Subtitle Extraction with Vision-Language Models](https://mlsanity.com/paper/2503.04058/) (2025)

## Built on by (most cited first, 5 of 5)

- [Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination](https://mlsanity.com/paper/2511.17490/) (2025)
- [SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQA](https://mlsanity.com/paper/2511.20190/) (2025)
- [PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents](https://mlsanity.com/paper/2605.10341/) (2026)
- [VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA](https://mlsanity.com/paper/2605.04870/) (2026)
- [Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction](https://mlsanity.com/paper/2609.25769/) (2026)

## Abstract (arXiv)

Video text-based visual question answering (Video TextVQA) aims to answer questions by explicitly reading and reasoning about the text involved in a video. Most works in this field follow a frame-level framework which suffers from redundant text entities and implicit relation modeling, resulting in limitations in both accuracy and efficiency. In this paper, we rethink the Video TextVQA task from an instance-oriented perspective and propose a novel model termed GAT (Gather and Trace). First, to obtain accurate reading result for each video text instance, a context-aggregated instance gathering module is designed to integrate the visual appearance, layout characteristics, and textual contents of the related entities into a unified textual representation. Then, to capture dynamic evolution of text in the video flow, an instance-focused trajectory tracing module is utilized to establish spatio-temporal relationships between instances and infer the final answer. Extensive experiments on several public Video TextVQA datasets validate the effectiveness and generalization of our framework. GAT outperforms existing Video TextVQA methods, video-language pretraining methods, and video large language models in both accuracy and inference speed. Notably, GAT surpasses the previous state-of-the-art Video TextVQA methods by 3.86% in accuracy and achieves ten times of faster inference speed than video large language models. The source code is available at https://github.com/zhangyan-ucas/GAT.