mlsanityData to 7 Oct 2026

VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA

arXiv 2605.04870 ↗First posted 6 May 2026cs.CVcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Video text-based visual question answering (Video TextVQA) aims to answer questions by reasoning over visual textual content appearing in videos. Despite the strong multimodal video understanding capabilities of recent Video-LLMs, their performance on existing Video TextVQA benchmarks remains limited. To better understand this gap, we conduct an upper-bound analysis through frame-wise question answering, counting a sample as correct if any frame yields the right answer, which significantly outperforms direct video-based inference and reveals a substantial performance gap. The results suggest that the primary bottleneck lies in the localization of key question-relevant evidence, rather than in reasoning capacity itself. Building on this insight, we propose a question-guided agent framework that explicitly anchors the relevant keyframes before answering. The approach operates effectively in a training-free setting and consistently surpasses direct video inference. With additional supervised fine-tuning (SFT) and reinforcement learning (RL), it achieves an average improvement of +12.12 in accuracy and +11.15 in ANLS across benchmarks, establishing new state-of-the-art results. Our study underscores the critical role of explicit keyframe anchoring for advancing Video TextVQA. The code will be publicly released.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 38 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

32 of 38 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2025Qwen2.5-VL Technical Reportcited by 5,418
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  12. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
20 more
  1. 2025Group Sequence Policy Optimizationcited by 788
  2. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  3. 2024InternLM2 Technical Reportcited by 391
  4. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  5. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  6. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  7. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  8. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  9. 2022DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spottingcited by 74
  10. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
  11. 2023A Large Cross-Modal Video Retrieval Dataset with Reading Comprehensioncited by 31
  12. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  13. 2023Reading Between the Lanes: Text VideoQA on the Roadcited by 23
  14. 2025MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarioscited by 18
  15. 2024GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matchingcited by 12
  16. 2025AdsQA: Towards Advertisement Video Understandingcited by 12
  17. 2024Track the Answer: Extending TextVQA from Image to Video with Spatio-Temporal Cluescited by 6
  18. 2025Gather and Trace: Rethinking Video TextVQA from an Instance-oriented Perspectivecited by 5
  19. 2025GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarkingcited by 4
  20. 2025SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQAcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VideoMultiAgents: A Multi-Agent Framework for Video Question Answering
  2. 2025Agentic Keyframe Search for Video Question Answering
  3. 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning
  4. 2026VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority
  5. 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validation
  6. 2026AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents
  7. 2026ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
  8. 2025MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering