mlsanityData to 14 Oct 2026

Unified Driving Tokens: Representation- and Geometry-Guided Discrete Tokenizer for Driving World Models and Planning

arXiv 2606.01935 ↗First posted 1 Jun 2026cs.CVcited by 1

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Discrete visual tokens should provide a compact representation for both token-based world modeling and planning in autonomous driving. However, most tokenizers are inherited from image generation and are optimized mainly for pixel reconstruction, which may leave a gap between what is easy to generate and what is useful to decode for driving decisions. We present a representation-guided and geometry-enhanced tokenizer that learns discrete tokens under joint supervision. The tokenizer aligns its discrete bottleneck with a frozen DINO feature space through feature decoding, while preserving appearance via RGB reconstruction with perceptual and adversarial losses. To inject geometric state-related cues, we add adjacent-frame depth and relative-pose supervision during training and stabilize joint objectives with multi-codebook quantization. We evaluate the same learned tokens with a lightweight planning readout and a GPT-style next-token world model. Experiments on NAVSIM show improved reconstruction fidelity and representation consistency, competitive planning performance under a fixed decoder, and better generative quality under matched settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 38 of its 41 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

38 of 41 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving11 Feb
  2. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
  3. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  4. 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models17 Jul
  5. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
  6. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  7. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  8. 2025A Survey of World Models for Autonomous Driving20 Jan
  9. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
  10. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  11. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  12. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun

Other references · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  10. 2025DINOv3cited by 1,414
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  12. 2022Planning-oriented Autonomous Drivingcited by 1,087
14 more
  1. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  2. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  4. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  5. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  6. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  7. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  8. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  9. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 153
  10. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
  11. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
  12. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  13. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
  14. 2025DVGT: Driving Visual Geometry Transformercited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026DriveTok: 3D Driving Scene Tokenization for Unified Multi-View Reconstruction and Understanding
  2. 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Driving
  3. 2025Efficient Multi-Camera Tokenization with Triplanes for End-to-End Driving
  4. 2024Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?
  5. 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
  6. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
  7. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model
  8. 2025FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning