mlsanityData to 14 Oct 2026

TARDIS STRIDE: A Spatio-Temporal Road Image Dataset and World Model for Autonomy

arXiv 2506.11302 ↗First posted 12 Jun 2025cs.CV cs.AIcited by 0

On the map: World models → World models for autonomous driving

Abstract (arXiv)

World models aim to simulate environments and enable effective agent behavior. However, modeling real-world environments presents unique challenges as they dynamically change across both space and, crucially, time. To capture these composed dynamics, we introduce a Spatio-Temporal Road Image Dataset for Exploration (STRIDE) permuting 360-degree panoramic imagery into rich interconnected observation, state and action nodes. Leveraging this structure, we can simultaneously model the relationship between egocentric views, positional coordinates, and movement commands across both space and time. We benchmark this dataset via TARDIS, a transformer-based generative world model that integrates spatial and temporal dynamics through a unified autoregressive framework trained on STRIDE. We demonstrate robust performance across a range of agentic tasks such as controllable photorealistic image synthesis, instruction following, autonomous self-control, and state-of-the-art georeferencing. These results suggest a promising direction towards sophisticated generalist agents--capable of understanding and manipulating the spatial and temporal aspects of their material environments--with enhanced embodied reasoning capabilities. Training code, datasets, and model checkpoints are made available at https://huggingface.co/datasets/Tera-AI/STRIDE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 48 of its 49 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

48 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  2. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  3. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep

Other references · 45

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  8. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  12. 2016Pixel Recurrent Neural Networkscited by 816
33 more
  1. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  2. 2024Genie: Generative Interactive Environmentscited by 747
  3. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  4. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  5. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  6. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  7. 2022MAGVIT: Masked Generative Video Transformercited by 346
  8. 2022Transformers are Sample-Efficient World Modelscited by 328
  9. 2024Diffusion Models Are Real-Time Game Enginescited by 296
  10. 2024Navigation World Modelscited by 272
  11. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  12. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  13. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 204
  14. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
  15. 2024Autoregressive Video Generation without Vector Quantizationcited by 186
  16. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  17. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 139
  18. 2022Diffusion Models for Video Prediction and Infillingcited by 135
  19. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  20. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 123
  21. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
  22. 2016Video Pixel Networkscited by 108
  23. 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
  24. 2021CCVS: Context-aware Controllable Video Synthesiscited by 69
  25. 2017Recurrent Environment Simulatorscited by 68
  26. 2020Transformation-based Adversarial Video Prediction on Large-Scale Datacited by 66
  27. 2021Predicting Video with VQVAEcited by 61
  28. 2023PIGEON: Predicting Image Geolocationscited by 56
  29. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  30. 2019Adversarial Video Generation on Complex Datasetscited by 33
  31. 2020Neural Game Engine: Accurate learning of generalizable forward models from pixelscited by 12
  32. 2024Statewide Visual Geolocalization in the Wildcited by 12
  33. 2019Action-conditioned Benchmarking of Robotic Video Prediction Models: a Comparative Studycited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GeoWAM: Visual Geometry World Action Models for Autonomous Driving
  2. 2025A Comprehensive Survey on World Models for Embodied AI
  3. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
  4. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
  5. 2026GeoWorldAD: Geometry World Action Model for Autonomous Driving
  6. 2026Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception
  7. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model
  8. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving