# MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration

arXiv: https://arxiv.org/abs/2006.08170 · HTML: https://mlsanity.com/paper/2006.08170/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 25 of 25)

- [Bridging State and History Representations: Understanding Self-Predictive RL](https://mlsanity.com/paper/2401.08898/) (2024)
- [Procedural Generalization by Planning with Self-Supervised World Models](https://mlsanity.com/paper/2111.01587/) (2021)
- [On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning](https://mlsanity.com/paper/2206.03271/) (2022)
- [Robust Policy Learning over Multiple Uncertainty Sets](https://mlsanity.com/paper/2202.07013/) (2022)
- [Context Shift Reduction for Offline Meta-Reinforcement Learning](https://mlsanity.com/paper/2311.03695/) (2023)
- [Model-based Lifelong Reinforcement Learning with Bayesian Exploration](https://mlsanity.com/paper/2210.11579/) (2022)
- [Offline Meta Reinforcement Learning with In-Distribution Online Adaptation](https://mlsanity.com/paper/2305.19529/) (2023)
- [Distributionally Adaptive Meta Reinforcement Learning](https://mlsanity.com/paper/2210.03104/) (2022)
- [Efficient Symbolic Policy Learning with Differentiable Symbolic Expression](https://mlsanity.com/paper/2311.02104/) (2023)
- [Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations](https://mlsanity.com/paper/2312.15909/) (2023)
- [MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control](https://mlsanity.com/paper/2101.00746/) (2021)
- [Accelerating exploration and representation learning with offline pre-training](https://mlsanity.com/paper/2304.00046/) (2023)
- [Explore and Control with Adversarial Surprise](https://mlsanity.com/paper/2107.07394/) (2021)
- [Continual Robot Learning using Self-Supervised Task Inference](https://mlsanity.com/paper/2309.04974/) (2023)
- [Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation](https://mlsanity.com/paper/2211.10861/) (2022)
- [Meta-Reinforcement Learning via Exploratory Task Clustering](https://mlsanity.com/paper/2302.07958/) (2023)
- [First-Explore, then Exploit: Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs](https://mlsanity.com/paper/2307.02276/) (2023)
- [BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shaping](https://mlsanity.com/paper/2409.05358/) (2024)
- [Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving](https://mlsanity.com/paper/2505.00031/) (2025)
- [Information-Theoretic Policy Pre-Training with Empowerment](https://mlsanity.com/paper/2510.05996/) (2025)
- [On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies](https://mlsanity.com/paper/2209.10072/) (2022)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure](https://mlsanity.com/paper/2405.00902/) (2024)
- [Learning Task Belief Similarity with Latent Dynamics for Meta-Reinforcement Learning](https://mlsanity.com/paper/2506.19785/) (2025)
- [Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agent](https://mlsanity.com/paper/2602.19837/) (2026)

## Abstract (arXiv)

Meta reinforcement learning (meta-RL) extracts knowledge from previous tasks and achieves fast adaptation to new tasks. Despite recent progress, efficient exploration in meta-RL remains a key challenge in sparse-reward tasks, as it requires quickly finding informative task-relevant experiences in both meta-training and adaptation. To address this challenge, we explicitly model an exploration policy learning problem for meta-RL, which is separated from exploitation policy learning, and introduce a novel empowerment-driven exploration objective, which aims to maximize information gain for task identification. We derive a corresponding intrinsic reward and develop a new off-policy meta-RL framework, which efficiently learns separate context-aware exploration and exploitation policies by sharing the knowledge of task inference. Experimental evaluation shows that our meta-RL method significantly outperforms state-of-the-art baselines on various sparse-reward MuJoCo locomotion tasks and more complex sparse-reward Meta-World tasks.