Data to 14 Oct 2026

Meta reinforcement learning as task inference

arXiv 1905.06424 ↗First posted 15 May 2019cs.LG cs.AI stat.MLcited by 77

Not on the map. Why?

Abstract (arXiv)

Humans achieve efficient learning by relying on prior knowledge about the structure of naturally occurring tasks. There is considerable interest in designing reinforcement learning (RL) algorithms with similar properties. This includes proposals to learn the learning algorithm itself, an idea also known as meta learning. One formal interpretation of this idea is as a partially observable multi-task RL problem in which task information is hidden from the agent. Such unknown task problems can be reduced to Markov decision processes (MDPs) by augmenting an agent's observations with an estimate of the belief about the task based on past experience. However estimating the belief state is intractable in most partially-observed MDPs. We propose a method that separately learns the policy and the task belief by taking advantage of various kinds of privileged information. Our approach can be very effective at solving standard meta-RL environments, as well as a complex continuous control environment with sparse rewards and requiring long-term memory.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 20; 2022: 15; 2023: 14; 2024: 11; 2025: 10; 2026: 5.

77 papers.

Sort:

By area:

2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 208
2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 53
2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 44
2022Hypernetworks in Meta-Reinforcement Learningcited by 42

38 more

2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
2021The Information Geometry of Unsupervised Reinforcement Learningcited by 37
2022Skill-based Meta-Reinforcement Learningcited by 37
2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 36
2021Tell me why! Explanations support learning relational and causal structurecited by 36
2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 36
2021Efficient Transformers in Reinforcement Learning using Actor-Learner Distillationcited by 35
2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learningcited by 29
2021Block Contextual MDPs for Continual Learningcited by 25
2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
2022Robust Policy Learning over Multiple Uncertainty Setscited by 18
2023Making Scalable Meta Learning Practicalcited by 17
2023Recurrent Hypernetworks are Surprisingly Strong in Meta-RLcited by 16
2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 15
2021Learning a subspace of policies for online adaptation in Reinforcement Learningcited by 14
2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 14
2021Meta Learning Black-Box Population-Based Optimizerscited by 13
2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
2022ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learningcited by 12
2022Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learningcited by 12
2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
2021Meta-Reinforcement Learning in Broad and Non-Parametric Environmentscited by 10
2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
2024X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learnercited by 10
2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
2021Meta-Reinforcement Learning by Tracking Task Non-stationaritycited by 8
2021MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerancecited by 8
2022Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Gamescited by 8
2024AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentationcited by 7
2022Meta Policy Learning for Cold-Start Conversational Recommendationcited by 6
2022Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approachcited by 6
2022Giving Feedback on Interactive Student Programs with Meta-Explorationcited by 6
2025Uncertainty Prioritized Experience Replaycited by 6
2022Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learningcited by 5
2022Meta-Reinforcement Learning with Self-Modifying Networkscited by 4

50 most cited of 77 All 77 (JSON)