# STRIDE-QA: Visual Question Answering Dataset for Spatiotemporal Reasoning in Urban Driving Scenes

arXiv: https://arxiv.org/abs/2508.10427 · HTML: https://mlsanity.com/paper/2508.10427/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 14 Aug 2025.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)
- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning](https://mlsanity.com/paper/2503.15558/) (2025)
- [Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving](https://mlsanity.com/paper/2410.22313/) (2024)
- [NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models](https://mlsanity.com/paper/2503.12772/) (2025)

## Built on by (most cited first, 9 of 9)

- [Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models](https://mlsanity.com/paper/2604.01259/) (2026)
- [DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving](https://mlsanity.com/paper/2605.23176/) (2026)
- [ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Driving](https://mlsanity.com/paper/2601.19582/) (2026)
- [D2-V2X: Depth-Driven Cooperative V2X Reasoning for Autonomous Driving](https://mlsanity.com/paper/2605.24098/) (2026)
- [Descriptor: Distance-Annotated Traffic Perception Question Answering (DTPQA)](https://mlsanity.com/paper/2511.13397/) (2025)
- [FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving](https://mlsanity.com/paper/2608.09591/) (2026)
- [Speed in the Blind Spot: An Interpretability Analysis of Dynamic Perception in VLMs for Autonomous Driving](https://mlsanity.com/paper/2609.37046/) (2026)
- [Towards Reliable Vision-Language Models for Autonomous Driving](https://mlsanity.com/paper/2610.01531/) (2026)
- [From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2610.04139/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) have been applied to autonomous driving to support decision-making in complex real-world scenarios. However, their training on static, web-sourced image-text pairs fundamentally limits the precise spatiotemporal reasoning required to understand and predict dynamic traffic scenes. We address this critical gap with STRIDE-QA, a large-scale visual question answering (VQA) dataset for physically grounded reasoning from an ego-centric perspective. Constructed from 100 hours of multi-sensor driving data in Tokyo, capturing diverse and challenging conditions, STRIDE-QA is the largest VQA dataset for spatiotemporal reasoning in urban driving, offering 16M QA pairs over 270K frames. Grounded by dense, automatically generated annotations including 3D bounding boxes, segmentation masks, and multi-object tracks, the dataset uniquely supports both object-centric and ego-centric reasoning through three novel QA tasks that require spatial localization and temporal prediction. Our benchmarks demonstrate that existing VLMs struggle significantly, with near-zero scores on prediction consistency. In contrast, VLMs fine-tuned on STRIDE-QA exhibit dramatic performance gains, achieving 55% success in spatial localization and 28% consistency in future motion prediction, compared to near-zero scores from general-purpose VLMs. Therefore, STRIDE-QA establishes a comprehensive foundation for developing more reliable VLMs for safety-critical autonomous systems.