ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
Not on the map. Why?
Abstract (arXiv)
Despite remarkable progress in multimodal understanding, current MLLMs still exhibit limitations in video text understanding, particularly when semantics emerge through the integration of temporally distributed textual cues across multiple frames. This perception challenge fundamentally differs from static image text understanding, yet existing datasets fail to capture: the vast majority of questions remain answerable from single frames, inadequately reflecting real-world video text comprehension demands. To address this, we present ViTexQA, a large-scale video-text QA dataset, and FrameThinker for robust multi-frame temporal reasoning. We build ViTexQA via a quality-controlled Chain-of-Thought (CoT) annotation pipeline boosted with temporal constraints; all its QA pairs demand cross-frame text fusion to solve, enforcing true temporal reliance. FrameThinker adopts two-stage training for explicit temporal modeling: CoT-Guided Supervised Fine-Tuning (SFT) generates frame-aware reasoning chains, followed by Temporally-grounded Reinforcement Learning (RL) optimized with multi-frame coherence rewards. Evaluations show our method outperforms SOTA baselines on ViTexQA, lifting ROUGE-L by 6.3%.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
44 of 61 references.
References · 44
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2019Towards VQA Models That Can Readcited by 1,718
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2023A Survey on Multimodal Large Language Modelscited by 909
32 more
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2025Seed1.5-VL Technical Reportcited by 373
- 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2019Scene Text Visual Question Answeringcited by 331
- 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
- 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
- 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
- 2020PP-OCR: A Practical Ultra Lightweight OCR Systemcited by 151
- 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 146
- 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
- 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 100
- 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
- 2025Kwai Keye-VL 1.5 Technical Reportcited by 73
- 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 44
- 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
- 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 32
- 2023Reading Between the Lanes: Text VideoQA on the Roadcited by 23
- 2022Watching the News: Towards VideoQA Models that can Readcited by 20
- 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
- 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
- 2025MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarioscited by 18
- 2025Video Summarization with Large Language Modelscited by 16
- 2025VidText: Towards Comprehensive Evaluation for Video Text Understandingcited by 11
- 2024TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignmentcited by 10
- 2023Bridging Synthetic and Real Worlds for Pre-training Scene Text Detectorscited by 9
- 2024Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Studycited by 7
- 2024BQA: Body Language Question Answering Dataset for Video Large Language Modelscited by 3
- 2026CodePercept: Code-Grounded Visual STEM Perception for MLLMscited by 3
- 2025Describe What You See with Multimodal Large Language Models to Enhance Video Recommendationscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024VidCtx: Context-aware Video Question Answering with Image Models
- 2024Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
- 2023Understanding Video Scenes through Text: Insights from Text-based Video Question Answering
- 2025TextVidBench: A Benchmark for Long Video Scene Text Understanding
- 2026VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA
- 2024Temporal Reasoning Transfer from Text to Video
- 2025LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering
- 2025Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames