mlsanityData to 7 Oct 2026

MAMBA: an Effective World Model Approach for Meta-Reinforcement Learning

arXiv 2403.09859 ↗First posted 14 Mar 2024cs.LGcited by 13

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

Meta-reinforcement learning (meta-RL) is a promising framework for tackling challenging domains requiring efficient exploration. Existing meta-RL algorithms are characterized by low sample efficiency, and mostly focus on low-dimensional task distributions. In parallel, model-based RL methods have been successful in solving partially observable MDPs, of which meta-RL is a special case. In this work, we leverage this success and propose a new model-based approach to meta-RL, based on elements from existing state-of-the-art model-based and meta-RL methods. We demonstrate the effectiveness of our approach on common meta-RL benchmark domains, attaining greater return with better sample efficiency (up to 15×) while requiring very little hyperparameter tuning. In addition, we validate our approach on a slate of more challenging, higher-dimensional domains, taking a step towards real-world generalizing agents.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 32 of its 42 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

32 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Mastering Diverse Domains through World Models10 Jan
  2. 2021Mastering Atari Games with Limited Data30 Oct
  3. 2020Mastering Atari with Discrete World Models5 Oct
  4. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  5. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  6. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  7. 2016Learning to reinforcement learncited by 369
  8. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  9. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  10. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  11. 2021Discovering and Achieving Goals via World Modelscited by 131
  12. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
15 more
  1. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 99
  2. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 95
  3. 2019Meta reinforcement learning as task inferencecited by 75
  4. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  5. 2022Evaluating Long-Term Memory in 3D Mazescited by 41
  6. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  7. 2020Exploration in Approximate Hyper-State Space for Meta Reinforcement Learningcited by 28
  8. 2020Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trialscited by 20
  9. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  10. 2021Model-based Meta Reinforcement Learning using Graph Structured Surrogate Modelscited by 16
  11. 2021Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixturecited by 12
  12. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  13. 2020Meta-Model-Based Meta-Policy Optimizationcited by 7
  14. 2022Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approachcited by 6
  15. 2022A model-based approach to meta-Reinforcement Learning: Transformers and tree searchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 1.

13 papers.

Sort:

By area:

  1. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  2. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  3. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 14
  4. 2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 12
  5. 2024MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Networkcited by 5
  6. 2024Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environmentscited by 4
  7. 2024CLIP-Mamba: CLIP Pretrained Mamba Models with OOD and Hessian Evaluationcited by 3
  8. 2025Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Taskscited by 3
  9. 2025Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligencecited by 1
  10. 2025Meta-reinforcement learning with minimum attentioncited by 0
  11. 2025Wavelet Predictive Representations for Non-Stationary Reinforcement Learningcited by 0
  12. 2025MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarioscited by 0
1 more
  1. 2026Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Robotic Control Using Model Based Meta Adaption
  2. 2023Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes
  3. 2021Offline Meta-Reinforcement Learning with Online Self-Supervision
  4. 2022Model-Based Offline Meta-Reinforcement Learning with Regularization
  5. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  6. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
  7. 2026Meta-Reinforcement Learning with Self-Reflection for Agentic Search
  8. 2022Discovered Policy Optimisation