mlsanityData to 7 Oct 2026

Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models

arXiv 2102.08291 ↗First posted 16 Feb 2021cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is a promising paradigm for solving sequential decision-making problems, but low data efficiency and weak generalization across tasks are bottlenecks in real-world applications. Model-based meta reinforcement learning addresses these issues by learning dynamics and leveraging knowledge from prior experience. In this paper, we take a closer look at this framework, and propose a new Thompson-sampling based approach that consists of a new model to identify task dynamics together with an amortized policy optimization step. We show that our model, called a graph structured surrogate model (GSSM), outperforms state-of-the-art methods in predicting environment dynamics. Additionally, our approach is able to obtain high returns, while allowing fast execution during deployment by avoiding test time policy gradient optimization.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 16 Feb 2021.

Builds on

44 of 63 references.

References · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2018World Modelscited by 1,444
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  8. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  10. 2017Convolutional Sequence to Sequence Learningcited by 550
  11. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  12. 2018Conditional Neural Processescited by 421
32 more
  1. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  2. 2016Learning to reinforcement learncited by 369
  3. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  4. 2018Neural Processescited by 307
  5. 2019Attentive Neural Processescited by 280
  6. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  7. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  8. 2018Stochastic Video Generation with a Learned Priorcited by 244
  9. 2017Advances in Variational Inferencecited by 236
  10. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  11. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  12. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  13. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  14. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  15. 2019Meta-Learning with Warped Gradient Descentcited by 98
  16. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 95
  17. 2019Exploring Model-based Planning with Policy Networkscited by 90
  18. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
  19. 2020Plannable Approximations to MDP Homomorphisms: Equivariance under Actionscited by 61
  20. 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
  21. 2017Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Controlcited by 50
  22. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  23. 2020Model-based Adversarial Meta-Reinforcement Learningcited by 32
  24. 2017Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processescited by 28
  25. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 24
  26. 2017Variational Memory Addressing in Generative Modelscited by 17
  27. 2019Meta-Learning surrogate models for sequential decision makingcited by 14
  28. 2018Learning Awareness Modelscited by 13
  29. 2016A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Gamescited by 9
  30. 2020Meta-Model-Based Meta-Policy Optimizationcited by 7
  31. 2018Learning and Inferring Movement with Deep Generative Modelcited by 2
  32. 2017Learning Image-Conditioned Dynamics Models for Control of Under-actuated Legged Millirobotscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 4; 2025: 5; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
  2. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
  3. 2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
  4. 2023Practical Equivariances via Relational Conditional Neural Processescited by 13
  5. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 13
  6. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 11
  7. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 11
  8. 2025Bridge the Inference Gaps of Neural Processes via Expectation Maximizationcited by 7
  9. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 3
  10. 2025PrivilegedDreamer: Explicit Imagination of Privileged Information for Rapid Adaptation of Learned Policiescited by 3
  11. 2022Graph Neural Networks for Relational Inductive Bias in Vision-based Deep Reinforcement Learning of Robot Controlcited by 2
  12. 2025Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learningcited by 2
4 more
  1. 2023On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQRcited by 1
  2. 2023Amortized Network Intervention to Steer the Excitatory Point Processescited by 0
  3. 2025VAO: Validation-Aligned Optimization for Cross-Task Generative Auto-Biddingcited by 0
  4. 2026Flow-based Policy With Distributional Reinforcement Learning in Trajectory Optimizationcited by 0