mlsanityData to 14 Oct 2026

Relationally Grounded Latent World Models for Autonomous Driving

arXiv 2609.24626 ↗First posted 21 Sep 2026cs.CV cs.ROcited by 0

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Latent world models learn predictive representations for autonomous driving, but the relational semantics these states preserve often remain implicit. We investigate whether traffic scene graphs can serve as privileged semantic supervision for latent world representations. Building on LAW, we construct actor-centric scene graphs from nuScenes 3D annotations, encode their serialized relational structure using a frozen text embedding model, and align the visual latent representations with this semantic target during training. We remove the supervision branch at inference, so it requires neither scene graphs nor 3D annotations and adds no test-time computation. On nuScenes, our method reduces average trajectory L2 error from 0.661 to 0.622 (5.9%) and collision rate from 0.456 to 0.217 (52.4%) relative to our retrained LAW baseline. It also outperforms an unstructured caption-style semantic target, supporting the benefit of explicit relational structure for latent world-model representation learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 26 of its 27 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

26 of 27 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving15 Jun
  2. 2026DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving11 May
  3. 2026DriveFuture: Future-Aware Latent World Models for Autonomous Driving10 May
  4. 2026DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Driving1 Apr
  5. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving25 Mar
  6. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving6 Feb
  7. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
  8. 2025Latent Chain-of-Thought World Modeling for End-to-End Driving11 Dec
  9. 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models17 Jul
  10. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
  11. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  12. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
2 more
  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  2. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep

Other references · 12

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  3. 2022Planning-oriented Autonomous Drivingcited by 1,087
  4. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  5. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  6. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  7. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  8. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
  9. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Drivingcited by 28
  10. 2025GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Drivingcited by 7
  11. 2026LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformerscited by 2
  12. 2026Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understandingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
  2. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
  3. 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving
  4. 2026LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model
  5. 2025CARScenes: Semantic VLM Dataset for Safe Autonomous Driving
  6. 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
  7. 2026CASCADE: A Spatio-Temporal-Causal Reasoning Representation and Dataset for Driving
  8. 2021Towards Learning Generalizable Driving Policies from Restricted Latent Representations