mlsanityData to 14 Oct 2026

TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning

arXiv 2509.13579 ↗First posted 16 Sep 2025cs.RO cs.AI cs.LGcited by 0

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

also in 2 more
Abstract (arXiv)

We present TreeIRL, a novel planner for autonomous driving that combines Monte Carlo tree search (MCTS) and inverse reinforcement learning (IRL) to achieve state-of-the-art performance in simulation and in real-world driving. The core idea is to use MCTS to find a promising set of safe candidate trajectories and a deep IRL scoring function to select the most human-like among them. We evaluate TreeIRL against both classical and state-of-the-art planners in large-scale simulations and on 500+ miles of real-world autonomous driving in the Las Vegas metropolitan area. Test scenarios include dense urban traffic, adaptive cruise control, cut-ins, and traffic lights. TreeIRL achieves the best overall performance, striking a balance between safety, progress, comfort, and human-likeness. To our knowledge, our work is the first demonstration of MCTS-based planning on public roads and underscores the importance of evaluating planners across a diverse set of metrics and in real-world environments. TreeIRL is highly extensible and could be further improved with reinforcement learning and imitation learning, providing a framework for exploring different combinations of classical and learning-based approaches to solve the planning bottleneck in autonomous driving.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 68 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

42 of 68 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research12 Oct
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec
  3. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients27 Sep
  4. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coach18 Aug

Other references · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  10. 2016End to End Learning for Self-Driving Carscited by 1,009
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  12. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
26 more
  1. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  2. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  3. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  4. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  5. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  6. 2019Causal Confusion in Imitation Learningcited by 288
  7. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  8. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 254
  9. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  10. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  11. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  12. 2019A Survey of Deep Learning Applications to Autonomous Vehicle Controlcited by 106
  13. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  14. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 97
  15. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 80
  16. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 69
  17. 2025Robust Autonomy Emerges from Self-Playcited by 67
  18. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  19. 2019Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Drivingcited by 52
  20. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  21. 2021DriverGym: Democratising Reinforcement Learning for Autonomous Drivingcited by 23
  22. 2022Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehiclescited by 22
  23. 2017The Value of Inferring the Internal State of Traffic Participants for Autonomous Freeway Drivingcited by 15
  24. 2023MBAPPE: MCTS-Built-Around Prediction for Planning Explicitlycited by 11
  25. 2023PBP: Path-based Trajectory Prediction for Autonomous Drivingcited by 6
  26. 2024Lab2Car: A Versatile Wrapper for Deploying Experimental Planners in Complex Real-world Environmentscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Driving in Real Life with Inverse Reinforcement Learning
  2. 2026Monte-Carlo Tree Search with Neural Network Guidance for Lane-Free Autonomous Driving
  3. 2025GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Prediction
  4. 2026Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study
  5. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving
  6. 2021Closing the Planning-Learning Loop with Application to Autonomous Driving
  7. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
  8. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments