Data to 7 Oct 2026

Meta-Reinforcement Learning of Structured Exploration Strategies

arXiv 1802.07245 ↗First posted 20 Feb 2018cs.LG cs.AI cs.NEcited by 128

Not on the map. Why?

Abstract (arXiv)

Exploration is a fundamental challenge in reinforcement learning (RL). Many of the current exploration methods for deep RL use task-agnostic objectives, such as information gain or bonuses based on state visitation. However, many practical applications of RL involve learning more than a single task, and prior tasks can be used to inform how exploration should be performed in new tasks. In this work, we explore how prior tasks can inform an agent about how to explore effectively in new situations. We introduce a novel gradient-based fast adaptation algorithm -- model agnostic exploration with structured noise (MAESN) -- to learn exploration strategies from prior experience. The prior experience is used both to initialize a policy and to acquire a latent exploration space that can inject structured stochasticity into a policy, producing exploration strategies that are informed by prior knowledge and are more effective than random action-space noise. We show that MAESN is more effective at learning exploration strategies when compared to prior meta-RL methods, RL without learned exploration strategies, and task-agnostic exploration methods. We evaluate our method on a variety of simulated tasks: locomotion with a wheeled robot, locomotion with a quadrupedal walker, and object manipulation.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 36; 2022: 26; 2023: 29; 2024: 13; 2025: 15; 2026: 6.

128 papers.

Sort:

By area:

2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
2023Large Language Models for Robotics: A Surveycited by 132
2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
2025Self-Adapting Language Modelscited by 71
2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
2022Winner Takes It All: Training Performant RL Populations for Combinatorial Optimizationcited by 57
2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52

38 more

2023In search of dispersed memories: Generative diffusion models are associative memory networkscited by 44
2022Transformers are Meta-Reinforcement Learnerscited by 39
2021Meta-Learning Dynamics Forecasting Using Task Inferencecited by 37
2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 37
2022Skill-based Meta-Reinforcement Learningcited by 36
2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
2021DACBench: A Benchmark Library for Dynamic Algorithm Configurationcited by 28
2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
2021Meta-Adaptive Nonlinear Control: Theory and Algorithmscited by 23
2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
2021Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platformscited by 22
2023Causal Reinforcement Learning: A Surveycited by 22
2021Wasserstein Unsupervised Reinforcement Learningcited by 21
2022What Matters For Meta-Learning Vision Regression Tasks?cited by 18
2021Unsupervised Reinforcement Learning in Multiple Environmentscited by 17
2023Making Scalable Meta Learning Practicalcited by 17
2021Model-based Meta Reinforcement Learning using Graph Structured Surrogate Modelscited by 16
2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
2021CoMPS: Continual Meta Policy Searchcited by 15
2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 15
2023SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learningcited by 13
2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
2021A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learningcited by 11
2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
2024Large Language Models as Agents in Two-Player Gamescited by 10
2024Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learningcited by 10
2025MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attackscited by 10
2021Linear Representation Meta-Reinforcement Learning for Instant Adaptationcited by 9
2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
2022Provable Generalization of Overparameterized Meta-learning Trained with SGDcited by 9
2022Meta Reinforcement Learning for Optimal Design of Legged Robotscited by 9
2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 8

50 most cited of 128 All 128 (JSON)