mlsanityData to 7 Oct 2026

Language Conditioned Traffic Generation

arXiv 2307.07947 ↗First posted 16 Jul 2023cs.CVcited by 86

Not on the map. Why?

Cited by 25 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Simulation forms the backbone of modern self-driving development. Simulators help develop, test, and improve driving systems without putting humans, vehicles, or their environment at risk. However, simulators face a major challenge: They rely on realistic, scalable, yet interesting content. While recent advances in rendering and scene reconstruction make great strides in creating static scene assets, modeling their layout, dynamics, and behaviors remains challenging. In this work, we turn to language as a source of supervision for dynamic traffic scene generation. Our model, LCTGen, combines a large language model with a transformer-based decoder architecture that selects likely map locations from a dataset of maps, and produces an initial traffic distribution, as well as the dynamics of each vehicle. LCTGen outperforms prior work in both unconditional and conditional traffic scene generation in terms of realism and fidelity. Code and video will be available at https://ariostgx.github.io/lctgen.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). Builds on 22 of its 82 references that are in the map. First posted on arXiv 16 Jul 2023.

Builds on

22 of 82 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  11. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  12. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
10 more
  1. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  3. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  4. 2016Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?cited by 194
  5. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  6. 2018Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsingcited by 105
  7. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
  8. 2018Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Datacited by 94
  9. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  10. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 27; 2025: 34; 2026: 20.

86 papers.

Sort:

By area:

  1. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  2. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  3. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
  4. 2025A Survey of World Models for Autonomous Drivingcited by 60
  5. 2023Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Modelscited by 59
  6. 2024A Survey for Foundation Models in Autonomous Drivingcited by 51
  7. 2023RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarioscited by 46
  8. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  9. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  10. 2023Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous drivingcited by 30
  11. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  12. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
38 more
  1. 2024Why Solving Multi-agent Path Finding with Large Language Model has not Succeeded Yetcited by 22
  2. 2024Causal Composition Diffusion Model for Closed-loop Traffic Generationcited by 22
  3. 2024Promptable Closed-loop Traffic Simulationcited by 20
  4. 2025SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Modelcited by 20
  5. 2024Probing Multimodal LLMs as World Models for Drivingcited by 19
  6. 2024Language-Driven Interactive Traffic Trajectory Generationcited by 19
  7. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHFcited by 18
  8. 2024Human-compatible driving partners through data-regularized self-play reinforcement learningcited by 17
  9. 2025Challenger: Affordable Adversarial Driving Video Generationcited by 16
  10. 2024Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Modelcited by 14
  11. 2025Generating Multimodal Driving Scenes via Next-Scene Predictioncited by 12
  12. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  13. 2024CaDRE: Controllable and Diverse Generation of Safety-Critical Driving Scenarios using Real-World Trajectoriescited by 11
  14. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  15. 2023Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latentcited by 10
  16. 2024UniGen: Unified Modeling of Initial Agent States and Trajectories for Generating Autonomous Driving Scenarioscited by 10
  17. 2024Words in Motion: Extracting Interpretable Control Vectors for Motion Transformerscited by 9
  18. 2024World Models: The Safety Perspectivecited by 9
  19. 2025Enhancing Autonomous Driving Safety with Collision Scenario Integrationcited by 9
  20. 2025LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critical Driving Scenarioscited by 9
  21. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  22. 2024Generative AI Agents in Autonomous Machines: A Safety Perspectivecited by 7
  23. 2025DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Modelscited by 7
  24. 2025Continuous Locomotive Crowd Behavior Generationcited by 7
  25. 2025Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Drivingcited by 7
  26. 2024Reviving Life on the Edge: Joint Score-Based Graph Generation of Rich Edge Attributescited by 6
  27. 2024Zero-shot Safety Prediction for Autonomous Robots with Foundation World Modelscited by 6
  28. 2024Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Surveycited by 6
  29. 2024Generating Traffic Scenarios via In-Context Learning to Learn Better Motion Plannercited by 6
  30. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  31. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  32. 2025CrashAgent: Crash Scenario Generation via Multi-modal Reasoningcited by 5
  33. 2025Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happencited by 5
  34. 2025AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Frameworkcited by 5
  35. 2025The Safety Challenge of World Models for Embodied AI Agents: A Reviewcited by 5
  36. 2025ChatBEV: Empowering Traffic Scene Understanding and Simulation via Vision-Language Modelcited by 4
  37. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  38. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3

50 most cited of 86 All 86 (JSON)