mlsanityData to 14 Oct 2026

Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving

arXiv 2409.18343 ↗First posted 26 Sep 2024cs.AIcited by 27

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

A major challenge in autonomous vehicle research is modeling agent behaviors, which has critical applications including constructing realistic and reliable simulations for off-board evaluation and forecasting traffic agents motion for onboard planning. While supervised learning has shown success in modeling agents across various domains, these models can suffer from distribution shift when deployed at test-time. In this work, we improve the reliability of agent behaviors by closed-loop fine-tuning of behavior models with reinforcement learning. Our method demonstrates improved overall performance, as well as improved targeted metrics such as collision rate, on the Waymo Open Sim Agents challenge. Additionally, we present a novel policy evaluation benchmark to directly assess the ability of simulated agents to measure the quality of autonomous vehicle planners and demonstrate the effectiveness of our approach on this new benchmark.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 34 of its 43 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

34 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research12 Oct
  2. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction7 Mar
  3. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec
  4. 2022TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving31 Mar

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  7. 2022Planning-oriented Autonomous Drivingcited by 1,087
  8. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  9. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  10. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  11. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  12. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
18 more
  1. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  2. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  3. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 190
  4. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  5. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 140
  6. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  7. 2022TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarioscited by 106
  8. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  9. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  10. 2023The Waymo Open Sim Agents Challengecited by 86
  11. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  12. 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
  13. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  14. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 33
  15. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  16. 2023Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023cited by 25
  17. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  18. 2022MolE: a molecular foundation model for drug discoverycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 18; 2026: 8.

27 papers.

Sort:

By area:

  1. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  2. 2025SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Modelcited by 20
  3. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  4. 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuningcited by 14
  5. 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
  6. 2025medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Supportcited by 9
  7. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  8. 2026Learning from Mistakes: Post-Training for Driving VLA with Takeover Datacited by 8
  9. 2026World Engine: Towards the Era of Post-Training for Autonomous Drivingcited by 8
  10. 2025SPACeR: Self-Play Anchoring with Centralized Reference Modelscited by 7
  11. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  12. 2025RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulationcited by 4
15 more
  1. 2025Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Drivingcited by 4
  2. 2026Human-like autonomy emerges from self-play and a pinch of human datacited by 4
  3. 2025SceneStreamer: Continuous Scenario Generation as Next Token Group Predictioncited by 2
  4. 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Drivingcited by 2
  5. 2026SCORP: Scene-Consistent Multi-agent Diffusion Planning with Stable Online Reinforcement Post-Training for Cooperative Drivingcited by 2
  6. 2026RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuningcited by 2
  7. 2025Navigating the Edge with the State-of-the-Art Insights into Corner Case Identification and Generation for Enhanced Autonomous Vehicle Safetycited by 1
  8. 2025LaViPlan : Language-Guided Visual Path Planning with RLVRcited by 1
  9. 2025Agent Fine-tuning through Distillation for Domain-specific LLMs in Microdomainscited by 1
  10. 2026World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planningcited by 1
  11. 2026CounterPlay: Counterfactual Post-Training for Self-Play Driving Policiescited by 1
  12. 2025LayerMix: Enhanced Data Augmentation through Fractal Integration for Robust Deep Learningcited by 0
  13. 2025CORTEX-AVD: A Framework for CORner Case Testing and EXploration in Autonomous Vehicle Developmentcited by 0
  14. 2025Constructing the Umwelt: Cognitive Planning through Belief-Intent Co-Evolutioncited by 0
  15. 2026MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving
  2. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
  3. 2021Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge
  4. 2025nuPlan-R: A Closed-Loop Planning Benchmark for Autonomous Driving via Reactive Multi-Agent Simulation
  5. 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving
  6. 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models
  7. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
  8. 2026Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty