mlsanityData to 14 Oct 2026

ReWorld: Representation Learning for World Action Models

arXiv 2606.27504 ↗First posted 25 Jun 2026cs.CVcited by 2

On the map: World models → World models for autonomous driving

Abstract (arXiv)

World Action Models (WAMs) unify future environment prediction with action generation for autonomous driving, yet existing approaches optimize only the final outputs, leaving intermediate representations as incidental byproducts. We present ReWorld, the first representation learning framework specifically designed for autonomous-driving WAMs. ReWorld explicitly optimizes the latent world-to-action pathway through three complementary mechanisms. First, it imposes future-predictive supervision on intermediate Video DiT states to encode temporal scene dynamics, enabling self-guided sampling and a roughly twofold convergence speedup. Second, it aligns Action DiT states with their attended video readouts so that the retrieved world information is retained in the representations used for planning. Third, it shapes the action space using geometrically close yet low-scoring hard negatives to separate the expert trajectory from nearby unsafe alternatives. ReWorld constructs supervision entirely from the WAM's own generation targets and attended features, requiring no external encoders or teacher models and introducing only 0.3% additional per-step training cost. Experiments show that ReWorld reduces FVD from 81.3 to 61.9 on nuScenes, improves closed-loop PDMS from 89.1 to 90.4 on NAVSIM without reinforcement learning or test-time scoring, and increases frozen linear-probe accuracy from 68.3% to 80.2% on UCF-101 action recognition. These results indicate that explicitly optimized representations are central to translating world knowledge into planning capability in WAMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 78 of its 83 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

78 of 83 references.

Closest ancestors · 23

Same direction, newest first

  1. 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr
  2. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving25 Mar
  3. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation16 Mar
  4. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
  5. 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
  6. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  7. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  8. 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
  9. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  10. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
  11. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving20 Mar
  12. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
11 more
  1. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
  2. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving2 Dec
  3. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  4. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  5. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
  6. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  7. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  8. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  9. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  10. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
  11. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun

Other references · 55

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2013Auto-Encoding Variational Bayescited by 4,053
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  10. 2022Planning-oriented Autonomous Drivingcited by 1,087
  11. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  12. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
38 more
  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  2. 2024Genie: Generative Interactive Environmentscited by 747
  3. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  4. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  5. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  6. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  7. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  8. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  9. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  10. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  11. 2025Diffusion Transformers with Representation Autoencoderscited by 295
  12. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  13. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  14. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
  15. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  16. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
  17. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
  18. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
  19. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
  20. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  21. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  22. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
  23. 2025Latent Diffusion Model without Variational Autoencodercited by 88
  24. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
  25. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 79
  26. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
  27. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
  28. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 45
  29. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 36
  30. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 36
  31. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
  32. 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32
  33. 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 31
  34. 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generationcited by 28
  35. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  36. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 22
  37. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 18
  38. 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Modelscited by 17

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
  2. 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
  3. 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
  4. 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
  5. 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
  6. 2026Rethinking Representations for World-Action Modeling
  7. 2026DA-WAM: Decision-Aligned Future Latents for Driving World Models
  8. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving