mlsanityData to 7 Oct 2026

Trajeglish: Traffic Modeling as Next-Token Prediction

arXiv 2312.04535 ↗First posted 7 Dec 2023cs.LG cs.ROcited by 59

Not on the map. Why?

Abstract (arXiv)

A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. In pursuit of this functionality, we apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Using a simple data-driven tokenization scheme, we discretize trajectories to centimeter-level resolution using a small vocabulary. We then model the multi-agent sequence of discrete motion tokens with a GPT-like encoder-decoder that is autoregressive in time and takes into account intra-timestep interaction between agents. Scenarios sampled from our model exhibit state-of-the-art realism; our model tops the Waymo Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%. We ablate our modeling choices in full autonomy and partial autonomy settings, and show that the representations learned by our model can quickly be adapted to improve performance on nuScenes. We additionally evaluate the scalability of our model with respect to parameter count and dataset size, and use density estimates from our model to quantify the saliency of context length and intra-timestep interaction for the traffic modeling task.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

23 of 29 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2017Decoupled Weight Decay Regularizationcited by 1,722
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  12. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
11 more
  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  2. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  3. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  4. 2016Using the Output Embedding to Improve Language Modelscited by 266
  5. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  6. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  7. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  8. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  9. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  10. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  11. 2019FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Networkcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 29; 2026: 22.

59 papers.

Sort:

By area:

  1. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  2. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  3. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  4. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  5. 2023SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversariescited by 36
  6. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  7. 2024Promptable Closed-loop Traffic Simulationcited by 20
  8. 2025Building reliable sim driving agents by scaling self-playcited by 20
  9. 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
  10. 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuningcited by 14
  11. 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
  12. 2025Trajectory Prediction Meets Large Language Models: A Surveycited by 13
38 more
  1. 2025Generating Multimodal Driving Scenes via Next-Scene Predictioncited by 12
  2. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  3. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  4. 2025Enhancing Autonomous Driving Safety with Collision Scenario Integrationcited by 9
  5. 2025TrajTok: Technical Report for 2025 Waymo Open Sim Agents Challengecited by 8
  6. 2024ModeSeq: Taming Sparse Multimodal Motion Prediction with Sequential Mode Modelingcited by 7
  7. 2025Continuous Locomotive Crowd Behavior Generationcited by 7
  8. 2025When Planners Meet Reality: How Learned, Reactive Traffic Agents Shift nuPlan Benchmarkscited by 7
  9. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  10. 2025Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrationscited by 6
  11. 2025DONUT: A Decoder-Only Model for Trajectory Predictioncited by 6
  12. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 6
  13. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  14. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  15. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
  16. 2025Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Drivingcited by 4
  17. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  18. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  19. 2025Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedbackcited by 3
  20. 2025nuPlan-R: A Closed-Loop Planning Benchmark for Autonomous Driving via Reactive Multi-Agent Simulationcited by 3
  21. 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
  22. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  23. 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Drivingcited by 2
  24. 2026Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Predictioncited by 2
  25. 2026AutoWorld: Learning Multi-Agent Traffic Simulation with Self-Supervised World Modelscited by 2
  26. 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
  27. 2025RADE: Learning Risk-Adjustable Driving Environment via Multi-Agent Conditional Diffusioncited by 1
  28. 2026SaFeR: Safety-Critical Scenario Generation for Autonomous Driving Test via Feasibility-Constrained Token Resamplingcited by 1
  29. 2026VectorWorld: Efficient Streaming World Model via Diffusion Flow on Vector Graphscited by 1
  30. 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Modelcited by 1
  31. 2026Efficient Equivariant Transformer for Self-Driving Agent Modelingcited by 1
  32. 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
  33. 2025From Marginal to Joint Predictions: Evaluating Scene-Consistent Trajectory Prediction Approaches for Automated Drivingcited by 0
  34. 2025Noise-Aware Generative Microscopic Traffic Simulationcited by 0
  35. 2025Path Diffuser: Diffusion Model for Data-Driven Traffic Simulatorcited by 0
  36. 2026General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Mergingcited by 0
  37. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  38. 2026Improving Crash Frequency Prediction from Simulated Traffic Conflicts Using Machine Learning Based Microsimulationcited by 0

50 most cited of 59 All 59 (JSON)