mlsanityData to 7 Oct 2026

Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models

arXiv 2412.05334 ↗First posted 5 Dec 2024cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world. Inspired by large language models, tokenized multi-agent policies have recently become the state-of-the-art in traffic simulation. However, they are typically trained through open-loop behavior cloning, and thus suffer from covariate shift when executed in closed-loop during simulation. In this work, we present Closest Among Top-K (CAT-K) rollouts, a simple yet effective closed-loop fine-tuning strategy to mitigate covariate shift. CAT-K fine-tuning only requires existing trajectory data, without reinforcement learning or generative adversarial imitation. Concretely, CAT-K fine-tuning enables a small 7M-parameter tokenized traffic simulation policy to outperform a 102M-parameter model from the same model family, achieving the top spot on the Waymo Sim Agent Challenge leaderboard at the time of submission. The code is available at https://github.com/NVlabs/catk.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

37 of 49 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  7. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  8. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  9. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  10. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  11. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  12. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
25 more
  1. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  2. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  3. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 154
  4. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  5. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  6. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  7. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
  8. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  9. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
  10. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  11. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  12. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 59
  13. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  14. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  15. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  16. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 41
  17. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  18. 2023Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encodingcited by 32
  19. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  20. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  21. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  22. 2023Reinforcement Learning with Human Feedback for Realistic Traffic Simulationcited by 20
  23. 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
  24. 2024TrafficBots V1.5: Traffic Simulation via Conditional VAEs and Transformers with Relative Pose Encodingcited by 13
  25. 2023Hierarchical Imitation Learning for Stochastic Environmentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 18.

40 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  2. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  3. 2025Building reliable sim driving agents by scaling self-playcited by 20
  4. 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuningcited by 14
  5. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  6. 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modelingcited by 9
  7. 2025TrajTok: Technical Report for 2025 Waymo Open Sim Agents Challengecited by 8
  8. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  9. 2025TeraSim-World: Worldwide Safety-Critical Data Synthesis for End-to-End Autonomous Drivingcited by 6
  10. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policiescited by 6
  11. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 6
  12. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
28 more
  1. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
  2. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
  3. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  4. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  5. 2025Surprise Potential as a Measure of Interactivity in Driving Scenarioscited by 3
  6. 2025Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedbackcited by 3
  7. 2025nuPlan-R: A Closed-Loop Planning Benchmark for Autonomous Driving via Reactive Multi-Agent Simulationcited by 3
  8. 2025Autoregressive Meta-Actions for Unified Controllable Trajectory Generationcited by 2
  9. 2025Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generationcited by 2
  10. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  11. 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Drivingcited by 2
  12. 2026AutoWorld: Learning Multi-Agent Traffic Simulation with Self-Supervised World Modelscited by 2
  13. 2026SCORP: Scene-Consistent Multi-agent Diffusion Planning with Stable Online Reinforcement Post-Training for Cooperative Drivingcited by 2
  14. 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
  15. 2026Learning Rollout from Sampling:An R1-Style Tokenized Traffic Simulation Modelcited by 1
  16. 2026Efficient Equivariant Transformer for Self-Driving Agent Modelingcited by 1
  17. 2026Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Drivingcited by 1
  18. 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
  19. 2026OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teachercited by 1
  20. 2025Linear Memory SE(2) Invariant Attentioncited by 0
  21. 2025Noise-Aware Generative Microscopic Traffic Simulationcited by 0
  22. 2026DeCoR: Design and Control Co-Optimization for Urban Streets Using Reinforcement Learningcited by 0
  23. 2026Bridging Local Observation and Global Simulation in Closed-Loop Traffic Modelingcited by 0
  24. 2026ECoSim: Data Efficient Fine-Tuning for Controllable Traffic Simulationcited by 0
  25. 2026Controllable Sim Agents with Behavior Latentscited by 0
  26. 2026Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulationcited by 0
  27. 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 0
  28. 2026PhaseShift: Topology-Aware Data Harmonization and Model Consolidation Across Signalized Intersectionscited by 0