mlsanityData to 14 Oct 2026

TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents

arXiv 2507.01823 ↗First posted 2 Jul 2025cs.LG cs.ROcited by 1

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

We present a novel approach to knowledge transfer in model-based reinforcement learning, addressing the critical challenge of deploying large world models in resource-constrained environments. Our method efficiently distills a high-capacity multi-task agent (317M parameters) into a compact model (1M parameters) on the MT30 benchmark, significantly improving performance across diverse tasks. Our distilled model achieves a state-of-the-art normalized score of 28.45, surpassing the original 1M parameter model score of 18.93. This improvement demonstrates the ability of our distillation technique to capture and consolidate complex multi-task knowledge. We further optimize the distilled model through FP16 post-training quantization, reducing its size by ~50%. Our approach addresses practical deployment limitations and offers insights into knowledge representation in large world models, paving the way for more efficient and accessible multi-task reinforcement learning systems in robotics and other resource-constrained applications. Code available at https://github.com/dmytro-kuzmenko/td-mpc-opt.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 28 of its 33 references that are in the map. First posted on arXiv 2 Jul 2025.

Builds on

28 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Knowledge Transfer in Model-Based Reinforcement Learning Agents for Efficient Multi-Task Learning9 Jan
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  3. 2023Mastering Diverse Domains through World Models10 Jan
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  5. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  6. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
  7. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb

Other references · 21

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2017Mixed Precision Trainingcited by 1,156
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  10. 2017A Survey on Multi-Task Learningcited by 840
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
9 more
  1. 2018Learning Dexterous In-Hand Manipulationcited by 687
  2. 2015Policy Distillationcited by 440
  3. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  4. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  5. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  6. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  7. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  8. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  9. 2019Distilling Policy Distillationcited by 84

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPCcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KD-MARL: Resource-Aware Knowledge Distillation in Multi-Agent Reinforcement Learning
  2. 2026Knowledge Distillation for Large Language Models
  3. 2025Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective
  4. 2023IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control
  5. 2025AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes
  6. 2026Reinforcement-aware Knowledge Distillation for LLM Reasoning
  7. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents
  8. 2025Distilling Reinforcement Learning into Single-Batch Datasets