mlsanityData to 14 Oct 2026

Learning to Plan, Planning to Learn: Adaptive Hierarchical RL-MPC for Sample-Efficient Decision Making

arXiv 2512.17091 ↗First posted 18 Dec 2025cs.LG cs.AI cs.ROcited by 1

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

We propose a new approach for solving planning problems with a hierarchical structure, fusing reinforcement learning and MPC planning. Our formulation tightly and elegantly couples the two planning paradigms. It leverages reinforcement learning actions to inform the MPPI sampler, and adaptively aggregates MPPI samples to inform the value estimation. The resulting adaptive process leverages further MPPI exploration where value estimates are uncertain, and improves training robustness and the overall resulting policies. This results in a robust planning approach that can handle complex planning problems and easily adapts to different applications, as demonstrated over several domains, including race driving, modified Acrobot, and Lunar Lander with added obstacles. Our results in these domains show better data efficiency and overall performance in terms of both rewards and task success, with up to a 72% increase in success rate compared to existing approaches, as well as accelerated convergence (x2.1) compared to non-adaptive sampling.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 39 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

22 of 39 references.

References · 22

  1. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  2. 2016OpenAI Gymcited by 1,736
  3. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  7. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  9. 2019Human Motion Trajectory Prediction: A Surveycited by 232
  10. 2019Variational Inference MPC for Bayesian Model-based Reinforcement Learningcited by 39
  11. 2020Blending MPC & Value Function Approximation for Efficient Reinforcement Learningcited by 32
  12. 2021Ensemble Bootstrapping for Q-Learningcited by 31
10 more
  1. 2023Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flightcited by 19
  2. 2019Information Theoretic Model Predictive Q-Learningcited by 18
  3. 2024Dreaming to Assist: Learning to Align with Human Objectives for Shared Control in High-Speed Racingcited by 7
  4. 2019Model-based Lookahead Reinforcement Learningcited by 5
  5. 2021Infusing model predictive control into meta-reinforcement learning for mobile robots in dynamic environmentscited by 5
  6. 2025Unifying Model Predictive Path Integral Control, Reinforcement Learning, and Diffusion Models for Optimal Control and Planningcited by 3
  7. 2025TD-GRPC: Temporal Difference Learning with Group Relative Policy Constraint for Humanoid Locomotioncited by 3
  8. 2023Learning the References of Online Model Predictive Control for Urban Self-Drivingcited by 2
  9. 2025A KL-regularization Framework for Learning to Plan with Adaptive Priorscited by 2
  10. 2025Real-Time Gait Adaptation for Quadrupeds using Model Predictive Control and Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Deep Reinforcement-Learning-Guided Model Predictive Control for Preventing Overtakes in Autonomous Racingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GEM-MPC: Balancing Exploration and Exploitation through Expert-Guided Planning
  2. 2025MPCritic: A plug-and-play MPC architecture for reinforcement learning
  3. 2026Beyond Policy Alignment: Closing the Planning-Learning Loop for Robot Control with Learned World Models
  4. 2026HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization
  5. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning
  6. 2026Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization
  7. 2025Reinforcement Learning-based Dynamic Adaptation for Sampling-Based Motion Planning in Agile Autonomous Driving
  8. 2024Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks