mlsanityData to 14 Oct 2026

Learning Task Belief Similarity with Latent Dynamics for Meta-Reinforcement Learning

arXiv 2506.19785 ↗First posted 24 Jun 2025cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Meta-reinforcement learning requires utilizing prior task distribution information obtained during exploration to rapidly adapt to unknown tasks. The efficiency of an agent's exploration hinges on accurately identifying the current task. Recent Bayes-Adaptive Deep RL approaches often rely on reconstructing the environment's reward signal, which is challenging in sparse reward settings, leading to suboptimal exploitation. Inspired by bisimulation metrics, which robustly extracts behavioral similarity in continuous MDPs, we propose SimBelief-a novel meta-RL framework via measuring similarity of task belief in Bayes-Adaptive MDP (BAMDP). SimBelief effectively extracts common features of similar task distributions, enabling efficient task identification and exploration in sparse reward environments. We introduce latent task belief metric to learn the common structure of similar tasks and incorporate it into the specific task belief. By learning the latent dynamics across task distributions, we connect shared latent task belief features with specific task features, facilitating rapid task identification and adaptation. Our method outperforms state-of-the-art baselines on sparse reward MuJoCo and panda-gym tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 57 references that are in the map. First posted on arXiv 24 Jun 2025.

Builds on

39 of 57 references.

References · 39

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  8. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  9. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  10. 2019Challenges of Real-World Reinforcement Learningcited by 291
  11. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  12. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
27 more
  1. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  2. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  3. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  4. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  5. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  6. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  7. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  8. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  9. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  10. 2017Meta Learning Shared Hierarchiescited by 93
  11. 2019Meta reinforcement learning as task inferencecited by 77
  12. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  13. 2021panda-gym: Open-source goal-conditioned environments for robotic learningcited by 59
  14. 2021Towards Robust Bisimulation Metric Learningcited by 49
  15. 2018Unsupervised Meta-Learning for Reinforcement Learningcited by 39
  16. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  17. 2019What Can Learned Intrinsic Rewards Capture?cited by 36
  18. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35
  19. 2020Exploration in Approximate Hyper-State Space for Meta Reinforcement Learningcited by 28
  20. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  21. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 25
  22. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 24
  23. 2020Latent Skill Planning for Exploration and Transfercited by 19
  24. 2019Meta Reinforcement Learning with Task Embedding and Shared Policycited by 15
  25. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  26. 2023DEIR: Efficient and Robust Exploration through Discriminative-Model-Based Episodic Intrinsic Rewardscited by 7
  27. 2022Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Taskscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Task-Aware Exploration via a Predictive Bisimulation Metric
  2. 2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills
  3. 2025Meta-RL with Bayesian Linear Task Models
  4. 2026Pessimistic Latent Task-aware Optimization for Robust Offline Meta-Reinforcement Learning
  5. 2025Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning
  6. 2026Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning
  7. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  8. 2025TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments