# Reading Between the Lanes: Text VideoQA on the Road

arXiv: https://arxiv.org/abs/2307.03948 · HTML: https://mlsanity.com/paper/2307.03948/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 47 references that are in the map.
- First posted on arXiv 8 Jul 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://mlsanity.com/paper/2005.00200/) (2020)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Just Ask: Learning to Answer Questions from Millions of Narrated Videos](https://mlsanity.com/paper/2012.00451/) (2020)
- [COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images](https://mlsanity.com/paper/1601.07140/) (2016)
- [Revealing Single Frame Bias for Video-and-Language Learning](https://mlsanity.com/paper/2206.03428/) (2022)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [TAP: Text-Aware Pre-training for Text-VQA and Text-Caption](https://mlsanity.com/paper/2012.04638/) (2020)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)

## Built on by (most cited first, 23 of 23)

- [EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering](https://mlsanity.com/paper/2502.07411/) (2025)
- [MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios](https://mlsanity.com/paper/2505.21333/) (2025)
- [VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models](https://mlsanity.com/paper/2504.13122/) (2025)
- [Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Grounding](https://mlsanity.com/paper/2509.11866/) (2025)
- [VidText: Towards Comprehensive Evaluation for Video Text Understanding](https://mlsanity.com/paper/2505.22810/) (2025)
- [An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes](https://mlsanity.com/paper/2504.15270/) (2025)
- [IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams](https://mlsanity.com/paper/2605.27074/) (2026)
- [Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination](https://mlsanity.com/paper/2511.17490/) (2025)
- [Scene-Text Grounding for Text-Based Video Question Answering](https://mlsanity.com/paper/2409.14319/) (2024)
- [GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarking](https://mlsanity.com/paper/2505.22228/) (2025)
- [Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challenges](https://mlsanity.com/paper/2511.13261/) (2025)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [Understanding Video Scenes through Text: Insights from Text-based Video Question Answering](https://mlsanity.com/paper/2309.01380/) (2023)
- [Evaluating Small Vision-Language Models on Distance-Dependent Traffic Perception](https://mlsanity.com/paper/2510.08352/) (2025)
- [SFA: Scan, Focus, and Amplify toward Guidance-aware Answering for Video TextVQA](https://mlsanity.com/paper/2511.20190/) (2025)
- [Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models](https://mlsanity.com/paper/2512.11899/) (2025)
- [InterAct-Video: Reasoning-Rich Video QA for Urban Traffic](https://mlsanity.com/paper/2507.14743/) (2025)
- [MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillance](https://mlsanity.com/paper/2509.09730/) (2025)
- [ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering](https://mlsanity.com/paper/2606.24602/) (2026)
- [VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation](https://mlsanity.com/paper/2511.08348/) (2025)
- [Descriptor: Distance-Annotated Traffic Perception Question Answering (DTPQA)](https://mlsanity.com/paper/2511.13397/) (2025)
- [VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA](https://mlsanity.com/paper/2605.04870/) (2026)
- [ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement](https://mlsanity.com/paper/2608.28784/) (2026)

## Abstract (arXiv)

Text and signs around roads provide crucial information for drivers, vital for safe navigation and situational awareness. Scene text recognition in motion is a challenging problem, while textual cues typically appear for a short time span, and early detection at a distance is necessary. Systems that exploit such information to assist the driver should not only extract and incorporate visual and textual cues from the video stream but also reason over time. To address this issue, we introduce RoadTextVQA, a new dataset for the task of video question answering (VideoQA) in the context of driver assistance. RoadTextVQA consists of 3,222 driving videos collected from multiple countries, annotated with 10,500 questions, all based on text or road signs present in the driving videos. We assess the performance of state-of-the-art video question answering models on our RoadTextVQA dataset, highlighting the significant potential for improvement in this domain and the usefulness of the dataset in advancing research on in-vehicle support systems and text-aware multimodal question answering. The dataset is available at http://cvit.iiit.ac.in/research/projects/cvit-projects/roadtextvqa