Data to 14 Oct 2026

Meta-Q-Learning

arXiv 1910.00125 ↗First posted 30 Sep 2019cs.LG stat.MLcited by 62

Not on the map. Why?

Abstract (arXiv)

This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learning is competitive with state-of-the-art meta-RL algorithms if given access to a context variable that is a representation of the past trajectory. Second, a multi-task objective to maximize the average reward across the training tasks is an effective method to meta-train RL policies. Third, past data from the meta-training replay buffer can be recycled to adapt the policy on a new task using off-policy updates. MQL draws upon ideas in propensity estimation to do so and thereby amplifies the amount of available data for adaptation. Experiments on standard continuous-control benchmarks suggest that MQL compares favorably with the state of the art in meta-RL.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 20; 2022: 16; 2023: 12; 2024: 8; 2025: 3; 2026: 1.

62 papers.

Sort:

By area:

2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 93
2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 53
2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35
2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 19
2021Collect & Infer -- a fresh look at data-efficient Reinforcement Learningcited by 17

38 more

2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learningcited by 17
2021Safe Exploration by Solving Early Terminated MDPcited by 15
2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 15
2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 11
2025MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attackscited by 11
2022Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planningcited by 10
2023Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learningcited by 10
2023Cross-Episodic Curriculum for Transformer Agentscited by 10
2023Efficient Symbolic Policy Learning with Differentiable Symbolic Expressioncited by 10
2021Linear Representation Meta-Reinforcement Learning for Instant Adaptationcited by 9
2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
2024AlphaRouter: Quantum Circuit Routing with Reinforcement Learning and Tree Searchcited by 8
2020FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularizationcited by 7
2021Faster Deep Reinforcement Learning with Slower Online Networkcited by 7
2022A picture of the space of typical learnable taskscited by 7
2021On the Practical Consistency of Meta-Reinforcement Learning Algorithmscited by 6
2021Neural Stochastic Dual Dynamic Programmingcited by 6
2022PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representationscited by 6
2021Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learningcited by 5
2022Learning Task Embeddings for Teamwork Adaptation in Multi-Agent Reinforcement Learningcited by 5
2022Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Taskscited by 5
2022Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysiscited by 5
2022Time-Varying Propensity Score to Bridge the Gap between the Past and Presentcited by 5
2021Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularizationcited by 4
2021Hindsight Foresight Relabeling for Meta-Reinforcement Learningcited by 4
2021Reinforcement Learning In Two Player Zero Sum Simultaneous Action Gamescited by 4
2022Efficient Meta Reinforcement Learning for Preference-based Fast Adaptationcited by 4
2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skillscited by 4
2024Prospective Learning: Learning for a Dynamic Futurecited by 4
2022Meta Reinforcement Learning with Successor Feature Based Contextcited by 3
2024Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactionscited by 3
2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
2021Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learningcited by 2
2022Learning to learn online with neuromodulated synaptic plasticity in spiking neural networkscited by 2

50 most cited of 62 All 62 (JSON)