# Knowledge Reutilization in Meta-Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.18132 · HTML: https://mlsanity.com/paper/2606.18132/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 18 of 18)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning](https://mlsanity.com/paper/1910.08348/) (2019)
- [MetaMorph: Learning Universal Controllers with Transformers](https://mlsanity.com/paper/2203.11931/) (2022)
- [DeepDPM: Deep Clustering With an Unknown Number of Clusters](https://mlsanity.com/paper/2203.14309/) (2022)
- [Stick-Breaking Variational Autoencoders](https://mlsanity.com/paper/1605.06197/) (2016)
- [Hierarchically Decoupled Imitation for Morphological Transfer](https://mlsanity.com/paper/2003.01709/) (2020)
- [Nonparametric Variational Auto-encoders for Hierarchical Representation Learning](https://mlsanity.com/paper/1703.07027/) (2017)
- [Recurrent Hypernetworks are Surprisingly Strong in Meta-RL](https://mlsanity.com/paper/2309.14970/) (2023)
- [Robust MAML: Prioritization task buffer with adaptive learning process for model-agnostic meta-learning](https://mlsanity.com/paper/2103.08233/) (2021)
- [Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning](https://mlsanity.com/paper/2305.00286/) (2023)
- [A Survey on Deep Clustering: From the Prior Perspective](https://mlsanity.com/paper/2406.19602/) (2024)

## Abstract (arXiv)

Meta-reinforcement learning enables fast adaptation by extracting shared structure from related tasks, but existing end-to-end methods often couple task inference with embodiment-specific control. This coupling can obscure non-parametric task semantics, reduce sample efficiency, and limit cross-agent reuse. We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents. The framework uses a Bayesian non-parametric prior to organize latent task modes and a high-level policy to generate task-level magnitude guidance. To bridge reusable task knowledge with different embodiments, we introduce a semantic-magnitude interface and a lightweight temporal adaptor, which convert frozen meta-knowledge into temporally aligned subgoals for embodiment-specific low-level controllers. Experiments on multiple locomotion agents show that our framework reduces final-step tracking error by 94.75% -- 99.79% compared with recent state-of-the-art baselines and achieves comparable deployment performance with about 23.8% of their interaction data.