PhysWAM: Physically Consistent World Action Model for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 2 more
Abstract (arXiv)
World-action models (WAMs) jointly predict how a scene will evolve and how an agent should act, however joint generation alone does not necessarily impose a shared geometric constraint on these predictions. We present PhysWAM, a unified world-action model for autonomous driving that co-denoises multiview video, metric depth, and ego motion within a single flow-matching transformer. To ground world and action generation in measured scene geometry, we introduce Coupled Point Projection (CPP) that unprojects the generated depth into 3D points, transforms them using the generated SE(3) ego motion, and minimizes their distance to LiDAR points transformed using the recorded ego motion. This geometric constraint promotes physical consistency with the measured scene by jointly supervising generated depth and motion alongside their standard flow-matching objectives. At inference, trajectory selection relies only on a simple label-free consensus rule, with no learned scorer or simulator feedback. We evaluate PhysWAM across NAVSIM v1 and v2 planning, zero-shot closed-loop transfer, and future video and metric-depth prediction. Despite PhysWAM's simple selection procedure, it achieves strong planning performance and transfers zero-shot to unseen driving environments. It also generates accurate metric depth and temporally coherent video, with CPP improving both planning and depth prediction. Together, these results demonstrate that the geometric relationship between scene depth and ego motion provides a direct way to couple world and action generation within a simple unified model.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 75 of its 80 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
75 of 80 references.
Closest ancestors · 29
Same direction, newest first
- 2026GeoWAM: Visual Geometry World Action Models for Autonomous Driving24 Aug
- 2026DA-WAM: Decision-Aligned Future Latents for Driving World Models19 Aug
- 20264D-WAM: 4D Consistent World Modeling for Autonomous Driving10 Aug
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving7 Aug
- 2026GeoWorldAD: Geometry World Action Model for Autonomous Driving20 Jul
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving27 May
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning14 May
- 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving11 May
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Driving10 May
- 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr
- 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving3 Apr
- 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving6 Feb
17 more
- 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
- 2025GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control28 May
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
- 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving2 Dec
- 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
- 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations20 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 46
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
34 more
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
- 2026World Action Models are Zero-shot Policiescited by 465
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
- 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 249
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 240
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
- 2025Pseudo-Simulation for Autonomous Drivingcited by 150
- 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
- 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 71
- 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 67
- 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 54
- 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 33
- 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 30
- 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 26
- 2025UniFuture: A 4D Driving World Model for Future Generation and Perceptioncited by 22
- 2025DVGT: Driving Visual Geometry Transformercited by 21
- 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planningcited by 17
- 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Modelcited by 13
- 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 13
- 2026Beyond Imitation: Learning Safe End-to-End Autonomous Driving from Hard Negativescited by 8
- 2025ReCamDriving: LiDAR-Free Camera-Controlled Video Synthesis for Novel Trajectoriescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
- 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
- 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
- 2026ReWorld: Representation Learning for World Action Models
- 2026UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation
- 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World
- 2026MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving