mlsanityData to 7 Oct 2026

TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction

arXiv 2303.04116 ↗First posted 7 Mar 2023cs.RO cs.CVcited by 56

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Data-driven simulation has become a favorable way to train and test autonomous driving algorithms. The idea of replacing the actual environment with a learned simulator has also been explored in model-based reinforcement learning in the context of world models. In this work, we show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles. Existing data-driven traffic simulators are lacking configurability and scalability. To generate configurable behaviors, for each agent we introduce a destination as navigational information, and a time-invariant latent personality that specifies the behavioral style. To improve the scalability, we present a new scheme of positional encoding for angles, allowing all agents to share the same vectorized context and the use of an architecture based on dot-product attention. As a result, we can simulate all traffic participants seen in dense urban scenarios. Experiments on the Waymo open motion dataset show TrafficBots can simulate realistic multi-agent behaviors and achieve good performance on the motion prediction task.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 36 of its 55 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

36 of 55 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
  2. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2020On Layer Normalization in the Transformer Architecturecited by 906
  7. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  8. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  9. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  11. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  12. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
22 more
  1. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  2. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  3. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  4. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  5. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  6. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  7. 2019Multiple Futures Predictioncited by 178
  8. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  9. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  10. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  11. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  12. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  13. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 112
  14. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  15. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  16. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  17. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  18. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  19. 2022Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Modelscited by 27
  20. 2018Learning from Demonstration in the Wildcited by 25
  21. 2021GRIT: Fast, Interpretable, and Verifiable Goal Recognition with Learned Decision Trees for Autonomous Drivingcited by 19
  22. 2021AIR2 for Interaction Predictioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 17; 2025: 17; 2026: 14.

56 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  3. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  4. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  5. 2023The Waymo Open Sim Agents Challengecited by 86
  6. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
  7. 20253D and 4D World Modeling: A Surveycited by 64
  8. 2025A Survey of World Models for Autonomous Drivingcited by 60
  9. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  10. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
  11. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  12. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
38 more
  1. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  2. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  3. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  4. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  5. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  6. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  7. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  8. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learningcited by 26
  9. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
  10. 2024Promptable Closed-loop Traffic Simulationcited by 20
  11. 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latentscited by 19
  12. 2025Object-Centric World Model for Language-Guided Manipulationcited by 16
  13. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  14. 2024TrafficBots V1.5: Traffic Simulation via Conditional VAEs and Transformers with Relative Pose Encodingcited by 13
  15. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  16. 2024World Models: The Safety Perspectivecited by 9
  17. 2025DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Modelscited by 7
  18. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  19. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  20. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
  21. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  22. 2025Beyond Simulation: Benchmarking World Models for Planning and Causality in Autonomous Drivingcited by 3
  23. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  24. 2025Context and Diversity Matter: The Emergence of In-Context Learning in World Modelscited by 3
  25. 2024How Could Generative AI Support Compliance with the EU AI Act? A Review for Safe Automated Driving Perceptioncited by 2
  26. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  27. 2026Enactor: From Traffic Simulators to Surrogate World Modelscited by 2
  28. 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
  29. 2026ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Drivingcited by 2
  30. 2026PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rolloutcited by 2
  31. 2025RADE: Learning Risk-Adjustable Driving Environment via Multi-Agent Conditional Diffusioncited by 1
  32. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulationcited by 1
  33. 2026Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Drivingcited by 1
  34. 2026TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scalecited by 1
  35. 2024Vectorized Representation Dreamer (VRD): Dreaming-Assisted Multi-Agent Motion-Forecastingcited by 0
  36. 2024Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehiclescited by 0
  37. 2025Benchmarking Autonomous Driving Planners Across Leaderboards: A Unified CARLA-Based Evaluationcited by 0
  38. 2026Conditional Flow Matching for Continuous Anomaly Detection in Autonomous Driving on a Manifold-Aware Spectral Spacecited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research
  2. 2026AutoWorld: Learning Multi-Agent Traffic Simulation with Self-Supervised World Models
  3. 2025Path Diffuser: Diffusion Model for Data-Driven Traffic Simulator
  4. 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning
  5. 2024Diffusion-Based Environment-Aware Trajectory Prediction
  6. 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles
  7. 2026ForSim: Stepwise Forward Simulation for Traffic Policy Fine-Tuning
  8. 2024Versatile Behavior Diffusion for Generalized Traffic Agent Simulation