mlsanityData to 7 Oct 2026

Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

arXiv 2010.01062 ↗First posted 2 Oct 2020cs.LG cs.AI stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep. One way to learn such behaviour is via meta-learning. Many existing methods however rely on dense rewards for meta-training, and can fail catastrophically if the rewards are sparse. Without a suitable reward signal, the need for exploration during meta-training is exacerbated. To address this, we propose HyperX, which uses novel reward bonuses for meta-training to explore in approximate hyper-state space (where hyper-states represent the environment state and the agent's task belief). We show empirically that HyperX meta-learns better task-exploration and adapts more successfully to new tasks than existing methods.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 4; 2023: 8; 2024: 7; 2025: 3; 2026: 1.

28 papers.

Sort:

By area:

  1. 2024Automated Design of Agentic Systemscited by 360
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  3. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  4. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  5. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  6. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  7. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  8. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 13
  9. 2024Active Inference as a Model of Agencycited by 12
  10. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  11. 2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 12
  12. 2022Distributionally Adaptive Meta Reinforcement Learningcited by 11
16 more
  1. 2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
  2. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  3. 2021MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Controlcited by 6
  4. 2022Planning to the Information Horizon of BAMDPs via Epistemic State Abstractioncited by 4
  5. 2023First-Explore, then Exploit: Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offscited by 4
  6. 2023Bayesian Exploration Networkscited by 3
  7. 2024Enabling Adaptive Agent Training in Open-Ended Simulators by Targeting Diversitycited by 3
  8. 2022BIMRL: Brain Inspired Meta Reinforcement Learningcited by 2
  9. 2024A Bayesian Solution To The Imitation Gapcited by 2
  10. 2023Never Explore Repeatedly in Multi-Agent Reinforcement Learningcited by 1
  11. 2025Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Reviewcited by 1
  12. 2021Knowledge is reward: Learning optimal exploration by predictive reward cashingcited by 0
  13. 2021Reinforcement Learning for Few-Shot Text Generation Adaptationcited by 0
  14. 2023RL3: Boosting Meta Reinforcement Learning via RL inside RL2cited by 0
  15. 2025Learning Task Belief Similarity with Latent Dynamics for Meta-Reinforcement Learningcited by 0
  16. 2026Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agentcited by 0