mlsanityData to 7 Oct 2026

Guided Conditional Diffusion for Controllable Traffic Simulation

arXiv 2210.17366 ↗First posted 31 Oct 2022cs.RO cs.AI cs.LGcited by 189

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make vehicles follow specific trajectories and traffic rules. On the other hand, data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. However, to the best of our knowledge, no traffic model offers both controllability and realism. In this work, we develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time (e.g., reach a goal or follow a speed limit) while maintaining realism and physical feasibility through enforced dynamics. The key technical idea is to leverage recent advances from diffusion modeling and differentiable logic to guide generated trajectories to meet rules defined using signal temporal logic (STL). We further extend guidance to multi-agent settings and enable interaction-based rules like collision avoidance. CTG is extensively evaluated on the nuScenes dataset for diverse and composite rules, demonstrating improvement over strong baselines in terms of the controllability-realism tradeoff.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 26 of its 37 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

26 of 37 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  12. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
14 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  2. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  3. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  4. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  5. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  6. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
  7. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  8. 2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 132
  9. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  10. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  11. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 63
  12. 2020Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methodscited by 51
  13. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
  14. 2022Semi-Supervised Trajectory-Feedback Controller Synthesis for Signal Temporal Logic Specificationscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 40; 2024: 47; 2025: 63; 2026: 39.

189 papers.

Sort:

By area:

  1. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  3. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  4. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  5. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  6. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  7. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  8. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  9. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  10. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  11. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  12. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
38 more
  1. 2023Compositional Foundation Models for Hierarchical Planningcited by 92
  2. 2023The Waymo Open Sim Agents Challengecited by 86
  3. 2024Constrained Synthesis with Projected Diffusion Modelscited by 83
  4. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  5. 2025Robust Autonomy Emerges from Self-Playcited by 64
  6. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  7. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  8. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
  9. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  10. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  11. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  12. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 41
  13. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  14. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  15. 2023Boosting Continuous Control with Consistency Policycited by 36
  16. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 36
  17. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  18. 2024Motion Planning Diffusion: Learning and Adapting Robot Motion Planning with Diffusion Modelscited by 32
  19. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 31
  20. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  21. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  22. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  23. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  24. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  25. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  26. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learningcited by 26
  27. 2023Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023cited by 25
  28. 2023trajdata: A Unified Interface to Multiple Human Trajectory Datasetscited by 25
  29. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  30. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  31. 2023SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Modelscited by 24
  32. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Drivingcited by 24
  33. 2023Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaversescited by 22
  34. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22
  35. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  36. 2023Reinforcement Learning with Human Feedback for Realistic Traffic Simulationcited by 20
  37. 2024Promptable Closed-loop Traffic Simulationcited by 20
  38. 2024AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusioncited by 20

50 most cited of 189 All 189 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Versatile Behavior Diffusion for Generalized Traffic Agent Simulation
  2. 2024Controllable Traffic Simulation through LLM-Guided Hierarchical Reasoning and Refinement
  3. 2024Diffusion-Based Environment-Aware Trajectory Prediction
  4. 2025Path Diffuser: Diffusion Model for Data-Driven Traffic Simulator
  5. 2024Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting
  6. 2024DragTraffic: Interactive and Controllable Traffic Scene Generation for Autonomous Driving
  7. 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles
  8. 2026ControlMap: Controllable High-Definition Map Generation for Traffic Scenario Simulation