ReWorld: Representation Learning for World Action Models
On the map: World models → World models for autonomous driving
Abstract (arXiv)
World Action Models (WAMs) unify future environment prediction with action generation for autonomous driving, yet existing approaches optimize only the final outputs, leaving intermediate representations as incidental byproducts. We present ReWorld, the first representation learning framework specifically designed for autonomous-driving WAMs. ReWorld explicitly optimizes the latent world-to-action pathway through three complementary mechanisms. First, it imposes future-predictive supervision on intermediate Video DiT states to encode temporal scene dynamics, enabling self-guided sampling and a roughly twofold convergence speedup. Second, it aligns Action DiT states with their attended video readouts so that the retrieved world information is retained in the representations used for planning. Third, it shapes the action space using geometrically close yet low-scoring hard negatives to separate the expert trajectory from nearby unsafe alternatives. ReWorld constructs supervision entirely from the WAM's own generation targets and attended features, requiring no external encoders or teacher models and introducing only 0.3% additional per-step training cost. Experiments show that ReWorld reduces FVD from 81.3 to 61.9 on nuScenes, improves closed-loop PDMS from 89.1 to 90.4 on NAVSIM without reinforcement learning or test-time scoring, and increases frozen linear-probe accuracy from 68.3% to 80.2% on UCF-101 action recognition. These results indicate that explicitly optimized representations are central to translating world knowledge into planning capability in WAMs.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 78 of its 83 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
78 of 83 references.
Closest ancestors · 23
Same direction, newest first
- 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr
- 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving25 Mar
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation16 Mar
- 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
- 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
- 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving20 Mar
- 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
11 more
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
- 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving2 Dec
- 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
- 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 55
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
38 more
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2024Genie: Generative Interactive Environmentscited by 747
- 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
- 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
- 2025Diffusion Transformers with Representation Autoencoderscited by 295
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 166
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
- 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
- 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
- 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
- 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
- 2025Latent Diffusion Model without Variational Autoencodercited by 88
- 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
- 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 79
- 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 78
- 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
- 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 45
- 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 36
- 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 36
- 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
- 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 32
- 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 31
- 2025One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generationcited by 28
- 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
- 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 22
- 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 18
- 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Modelscited by 17
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026VITAL: From Visual Motion to Precise Action14 Oct 2026
- 2026ReDrive: Shaping Representations with World Modeling for End-to-End Driving27 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
- 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
- 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
- 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
- 2026Rethinking Representations for World-Action Modeling
- 2026DA-WAM: Decision-Aligned Future Latents for Driving World Models
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving