mlsanityData to 14 Oct 2026

Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation

arXiv 2510.00212 ↗First posted 30 Sep 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents notable challenges. First, MAML relies on second-order gradient computations, leading to significant computational and memory overhead. Second, the nested structure of optimization increases the problem's complexity, making convergence to a global optimum more challenging. To overcome these limitations, we propose Directed-MAML, a novel task-directed meta-RL algorithm. Before the second-order gradient step, Directed-MAML applies an additional first-order task-directed approximation to estimate the effect of second-order gradients, thereby accelerating convergence to the optimum and reducing computational cost. Experimental results demonstrate that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed in the scenarios of CartPole-v1, LunarLander-v2 and two-vehicle intersection crossing. Furthermore, we show that task-directed approximation can be effectively integrated into other meta-learning algorithms, such as First-Order Model-Agnostic Meta-Learning (FOMAML) and Meta Stochastic Gradient Descent(Meta-SGD), yielding improved computational efficiency and convergence speed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 27 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

17 of 27 references.

References · 17

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2016OpenAI Gymcited by 1,736
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  5. 2019Meta-Learning with Implicit Gradientscited by 525
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  8. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  9. 2018How to train your MAMLcited by 309
  10. 2018Unsupervised Meta-Learning For Few-Shot Image Classificationcited by 75
  11. 2019Meta-Learning of Neural Architectures for Few-Shot Learningcited by 73
  12. 2020Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Taskscited by 51
5 more
  1. 2020Meta-learning for Few-shot Natural Language Processing: A Surveycited by 42
  2. 2022Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learningcited by 22
  3. 2019Meta-learning algorithms for Few-Shot Computer Visioncited by 3
  4. 2021Meta-Learning with MAML on Treescited by 2
  5. 2024MAML MOT: Multiple Object Tracking based on Meta-Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Constrained Meta Agnostic Reinforcement Learning
  2. 2026From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning
  3. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning
  4. 2025Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning
  5. 2024Theoretical Analysis of Meta Reinforcement Learning: Generalization Bounds and Convergence Guarantees
  6. 2024Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator
  7. 2022Discovered Policy Optimisation
  8. 2025TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments