mlsanityData to 14 Oct 2026

EfficientTDMPC: Improved MPC Objectives for Sample-Efficient Continuous Control

arXiv 2605.16692 ↗First posted 15 May 2026cs.LG cs.AI cs.ROcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

We introduce EfficientTDMPC, a sample-efficient model-based reinforcement learning method for continuous control built on the TD-MPC family of algorithms. Central to this family is a planner that aims to find an action sequence that maximizes the estimated return. The return is estimated using a learned model and value networks, each of which can introduce error. EfficientTDMPC introduces three contributions that improve performance by aiming to reduce this error. First, we introduce a multi-horizon planning objective that evaluates the value at different rollout depths and averages them. Second, to our knowledge we are the first to train a value-equivalent dynamics ensemble. Our improved objective then averages over rollouts from multiple dynamics heads. Third, we add pessimistic reanalyze for tasks that can terminate early. Applying our contributions to a recent baseline (BMPC) yields EfficientTDMPC, which to our knowledge is the new state of the art in sample efficiency on HumanoidBench and the DeepMind Control Suite, reaching BMPC's final aggregated performance using 57% fewer environment steps.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 41 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

30 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026The Surprising Difficulty of Search in Model-Based Reinforcement Learning29 Jan
  2. 2025Bootstrap Off-policy with World Model1 Nov
  3. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learning21 Feb
  4. 2024EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data1 Mar

Other references · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2018DeepMind Control Suitecited by 956
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  10. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  11. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  12. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
14 more
  1. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  2. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  3. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  4. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  5. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  6. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
  7. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  8. 2025Bootstrapped Model Predictive Controlcited by 16
  9. 2025TD-M(PC)2: Improving Temporal Difference MPC Through Policy Constraintcited by 14
  10. 2022Epistemic Monte Carlo Tree Searchcited by 7
  11. 2024Value Improved Actor Critic Algorithmscited by 7
  12. 2024Masked Generative Priors Improve World Models Sequence Modelling Capabilitiescited by 7
  13. 2025Twice Sequential Monte Carlo for Tree Searchcited by 3
  14. 2026Debiased Model-based Representations for Sample-efficient Continuous Controlcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026GEM-MPC: Balancing Exploration and Exploitation through Expert-Guided Planningcited by 0
  2. 2026Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPCcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning to Optimize in Model Predictive Control
  2. 2026Beyond Policy Alignment: Closing the Planning-Learning Loop for Robot Control with Learned World Models
  3. 2026WIMLE: Uncertainty-Aware World Models with IMLE for Sample-Efficient Continuous Control
  4. 2024Faster Model Predictive Control via Self-Supervised Initialization Learning
  5. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control
  6. 2026Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
  7. 2026Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion
  8. 2025XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning