# Some Considerations on Learning to Explore via Meta-Reinforcement Learning

arXiv: https://arxiv.org/abs/1803.01118 · HTML: https://mlsanity.com/paper/1803.01118/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2018.

## Built on by (most cited first, 30 of 44)

- [Human-Timescale Adaptation in an Open-Ended Task Space](https://mlsanity.com/paper/2301.07608/) (2023)
- [Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.07572/) (2025)
- [Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought](https://mlsanity.com/paper/2501.04682/) (2025)
- [A Survey of Exploration Methods in Reinforcement Learning](https://mlsanity.com/paper/2109.00157/) (2021)
- [Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data](https://mlsanity.com/paper/2412.07762/) (2024)
- [XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2312.12044/) (2023)
- [Bootstrapped Meta-Learning](https://mlsanity.com/paper/2109.04504/) (2021)
- [AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents](https://mlsanity.com/paper/2310.09971/) (2023)
- [Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents](https://mlsanity.com/paper/2102.02926/) (2021)
- [Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning](https://mlsanity.com/paper/2206.10442/) (2022)
- [Watch, Try, Learn: Meta-Learning from Demonstrations and Reward](https://mlsanity.com/paper/1906.03352/) (2019)
- [Meta-RL Induces Exploration in Language Agents](https://mlsanity.com/paper/2512.16848/) (2025)
- [In-context Exploration-Exploitation for Reinforcement Learning](https://mlsanity.com/paper/2403.06826/) (2024)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers](https://mlsanity.com/paper/2411.11188/) (2024)
- [Context Meta-Reinforcement Learning via Neuromodulation](https://mlsanity.com/paper/2111.00134/) (2021)
- [Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning](https://mlsanity.com/paper/2305.00286/) (2023)
- [Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learning](https://mlsanity.com/paper/2103.06386/) (2021)
- [Meta-Reinforcement Learning by Tracking Task Non-stationarity](https://mlsanity.com/paper/2105.08834/) (2021)
- [Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL](https://mlsanity.com/paper/2112.00901/) (2021)
- [Meta-Reinforcement Learning with Self-Reflection for Agentic Search](https://mlsanity.com/paper/2603.11327/) (2026)
- [On the Practical Consistency of Meta-Reinforcement Learning Algorithms](https://mlsanity.com/paper/2112.00478/) (2021)
- [Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation](https://mlsanity.com/paper/2106.13125/) (2021)
- [Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks](https://mlsanity.com/paper/2207.09071/) (2022)
- [An Investigation of the Bias-Variance Tradeoff in Meta-Gradients](https://mlsanity.com/paper/2209.11303/) (2022)
- [Hindsight Foresight Relabeling for Meta-Reinforcement Learning](https://mlsanity.com/paper/2109.09031/) (2021)
- [Meta-Reinforcement Learning via Exploratory Task Clustering](https://mlsanity.com/paper/2302.07958/) (2023)
- [First-Explore, then Exploit: Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs](https://mlsanity.com/paper/2307.02276/) (2023)
- [Variational Meta Reinforcement Learning for Social Robotics](https://mlsanity.com/paper/2206.03211/) (2022)
- [Meta Reinforcement Learning with Successor Feature Based Context](https://mlsanity.com/paper/2207.14723/) (2022)

## Abstract (arXiv)

We consider the problem of exploration in meta reinforcement learning. Two new meta reinforcement learning algorithms are suggested: E-MAML and E-RL2. Results are presented on a novel environment we call `Krazy World' and a set of maze environments. We show E-MAML and E-RL2 deliver better performance on tasks where exploration is important.