PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rollout
On the map: World models → World models for autonomous driving
Abstract (arXiv)
Local pedestrian-vehicle forecasting spans heterogeneous physical scales: pedestrians combine root locomotion with articulated motion, whereas vehicles are rigid bodies described by kinematic state and oriented extent. Existing road-agent forecasters typically omit pedestrian articulation, while pose forecasters leave vehicle futures outside the learned rollout. We introduce PV-WM, a history-only world model over structured post-perception tracks. It recurrently advances pedestrian root motion, 15-joint articulation, and learned vehicle states within a synchronized heterogeneous state. The generated pedestrian and vehicle chunks supply the next recurrent boundary; vehicle boxes are reconstructed from predicted center and heading with observed extent, and P-V geometry is recomputed after every transition. Relative to a matched one-shot complete-state predictor, recurrent execution reduces Root ADE by 12.7% and MPJPE by 14.8%. Feedback interventions show that later predictions depend on the content, temporal order, and pedestrian identity of generated articulation. Across 824 aligned Waymo contexts, with 797 providing valid future vehicle support, PV-WM reduces Root ADE by 5.2%, MPJPE by 7.6%, P-V distance error by 11.9%, and oriented-box closest-approach error by 5.8% relative to a validation-selected Modular Specialist. The single-network model uses 57.1% fewer parameters, 96.5% lower average FLOPs per local scene, and 25.5% lower measured p95 latency. PV-WM unifies this heterogeneous future state while preserving type-specific pedestrian and vehicle dynamics.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 22 of its 27 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
22 of 27 references.
Closest ancestors · 9
Same direction, newest first
- 2026Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout6 May
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction7 Mar
- 2022Motion Transformer with Global Intention Localization and Local Movement Refinement27 Sep
- 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networks12 Jul
Other references · 13
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2018World Modelscited by 1,513
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
- 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
- 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
- 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
- 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
- 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 30
- 2023Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecastingcited by 26
- 2025Waymo-3DSkelMo: A Multi-Agent 3D Skeletal Motion Dataset for Pedestrian Interaction Modeling in Autonomous Drivingcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2021Scene Transformer: A unified architecture for predicting multiple agent trajectories
- 2025SceneAware: Scene-Constrained Pedestrian Trajectory Prediction with LLM-Guided Walkability
- 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecasting
- 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives
- 2026PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction
- 2026Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles
- 2025Realistic pedestrian-driver interaction modelling using multi-agent RL with human perceptual-motor constraints
- 2022Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting