mlsanityData to 7 Oct 2026

Promptable Closed-loop Traffic Simulation

arXiv 2409.05863 ↗First posted 9 Sep 2024cs.CV cs.AI cs.ROcited by 20

Not on the map. Why?

Abstract (arXiv)

Simulation stands as a cornerstone for safe and efficient autonomous driving development. At its core a simulation system ought to produce realistic, reactive, and controllable traffic patterns. In this paper, we propose ProSim, a multimodal promptable closed-loop traffic simulation framework. ProSim allows the user to give a complex set of numerical, categorical or textual prompts to instruct each agent's behavior and intention. ProSim then rolls out a traffic scenario in a closed-loop manner, modeling each agent's interaction with other traffic participants. Our experiments show that ProSim achieves high prompt controllability given different user prompts, while reaching competitive performance on the Waymo Sim Agents Challenge when no prompt is given. To support research on promptable traffic simulation, we create ProSim-Instruct-520k, a multimodal prompt-scenario paired driving dataset with over 10M text prompts for over 520k real-world driving scenarios. We will release code of ProSim as well as data and labeling tools of ProSim-Instruct-520k at https://ariostgx.github.io/ProSim.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

18 of 20 references.

References · 18

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  3. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  4. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  5. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  6. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
  7. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
  8. 2023The Waymo Open Sim Agents Challengecited by 86
  9. 2023Language Conditioned Traffic Generationcited by 86
  10. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  11. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  12. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
6 more
  1. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  2. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  3. 2024Versatile Behavior Diffusion for Generalized Traffic Agent Simulationcited by 30
  4. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  5. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  6. 2023The 2nd Place Solution for 2023 Waymo Open Sim Agents Challengecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 8.

20 papers.

Sort:

By area:

  1. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  2. 2025SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Modelcited by 20
  3. 2024Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Modelcited by 14
  4. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  5. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  6. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  7. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  8. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  9. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  10. 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
  11. 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
  12. 2025Test Automation for Interactive Scenarios via Promptable Traffic Simulationcited by 1
8 more
  1. 2026Risk-Aware World Model Predictive Control for Generalizable End-to-End Autonomous Drivingcited by 1
  2. 2026Separating Diagnosis from Control: Auditable Policy Adaptation in Agent-Based Simulations with LLM-Based Diagnosticscited by 1
  3. 2025LLM-based Human-like Traffic Simulation for Self-driving Testscited by 0
  4. 2026General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Mergingcited by 0
  5. 2026ECoSim: Data Efficient Fine-Tuning for Controllable Traffic Simulationcited by 0
  6. 2026Agent-driven Long-tail Simulation for Autonomous Drivingcited by 0
  7. 2026Large Multimodal Agents for Intelligent Transportation Systems: Architectures, Evidence, and Deployment Challengescited by 0
  8. 2026One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulationcited by 0