mlsanityData to 7 Oct 2026

UniGen: Unified Modeling of Initial Agent States and Trajectories for Generating Autonomous Driving Scenarios

arXiv 2405.03807 ↗First posted 6 May 2024cs.RO cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

This paper introduces UniGen, a novel approach to generating new traffic scenarios for evaluating and improving autonomous driving software through simulation. Our approach models all driving scenario elements in a unified model: the position of new agents, their initial state, and their future motion trajectories. By predicting the distributions of all these variables from a shared global scenario embedding, we ensure that the final generated scenario is fully conditioned on all available context in the existing scene. Our unified modeling approach, combined with autoregressive agent injection, conditions the placement and motion trajectory of every new agent on all existing agents and their trajectories, leading to realistic scenarios with low collision rates. Our experimental results show that UniGen outperforms prior state of the art on the Waymo Open Motion Dataset.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 32 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

28 of 32 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  7. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  8. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  9. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  11. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  12. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
16 more
  1. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  2. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  3. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
  4. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  5. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  6. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  7. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 104
  8. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
  9. 2023The Waymo Open Sim Agents Challengecited by 86
  10. 2023Language Conditioned Traffic Generationcited by 86
  11. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  12. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  13. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  14. 2023Generating Driving Scenes with Diffusioncited by 16
  15. 2022StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Drivingcited by 14
  16. 2020Testing the Safety of Self-driving Vehicles by Simulating Perception and Predictioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 1.

10 papers.

  1. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 47
  2. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  3. 2025Building reliable sim driving agents by scaling self-playcited by 20
  4. 2025SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Modelcited by 20
  5. 2025Generating Multimodal Driving Scenes via Next-Scene Predictioncited by 12
  6. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  7. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  8. 2025Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safetycited by 1
  9. 2025CORTEX-AVD: A Framework for CORner Case Testing and EXploration in Autonomous Vehicle Developmentcited by 0
  10. 2026MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understandingcited by 0