mlsanityData to 7 Oct 2026

Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policies

arXiv 2310.16686 ↗First posted 25 Oct 2023cs.AI cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

While reinforcement learning has achieved remarkable successes in several domains, its real-world application is limited due to many methods failing to generalise to unfamiliar conditions. In this work, we consider the problem of generalising to new transition dynamics, corresponding to cases in which the environment's response to the agent's actions differs. For example, the gravitational force exerted on a robot depends on its mass and changes the robot's mobility. Consequently, in such cases, it is necessary to condition an agent's actions on extrinsic state information and pertinent contextual information reflecting how the environment responds. While the need for context-sensitive policies has been established, the manner in which context is incorporated architecturally has received less attention. Thus, in this work, we present an investigation into how context information should be incorporated into behaviour learning to improve generalisation. To this end, we introduce a neural network architecture, the Decision Adapter, which generates the weights of an adapter module and conditions the behaviour of an agent on the context information. We show that the Decision Adapter is a useful generalisation of a previously proposed architecture and empirically demonstrate that it results in superior generalisation performance compared to previous approaches in several environments. Beyond this, the Decision Adapter is more robust to irrelevant distractor variables than several alternative methods.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 76 of its 108 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

76 of 108 references.

References · 76

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2016OpenAI Gymcited by 1,684
  10. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  11. 2016HyperNetworkscited by 1,003
  12. 2022A Generalist Agentcited by 802
38 more
  1. 2019Solving Rubik's Cube with a Robot Handcited by 699
  2. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  3. 2018Learning Dexterous In-Hand Manipulationcited by 643
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2017Learning multiple visual domains with residual adapterscited by 553
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  8. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  9. 2017Robust Adversarial Reinforcement Learningcited by 439
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  11. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  13. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  14. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  15. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  16. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  17. 2018Graph networks as learnable physics engines for inference and controlcited by 242
  18. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  19. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  20. 2015Contextual Markov Decision Processescited by 213
  21. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  22. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  23. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  24. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  25. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  26. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  27. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  28. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  29. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  30. 2023A Tutorial on Meta-Reinforcement Learningcited by 102
  31. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  32. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 95
  33. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 86
  34. 2022MetaMorph: Learning Universal Controllers with Transformerscited by 78
  35. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  36. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 64
  37. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  38. 2019Wasserstein Robust Reinforcement Learningcited by 50

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 9; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalizationcited by 16
  2. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  3. 2024Hierarchical Transformers are Efficient Meta-Reinforcement Learnerscited by 6
  4. 2025Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalizationcited by 5
  5. 2025Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policycited by 4
  6. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learningcited by 2
  7. 2024GRAM: Generalization in Deep RL with a Robust Adaptation Modulecited by 2
  8. 2025Make Haste Slowly: A Theory of Emergent Structured Mixed Selectivity in Feature Learning ReLU Networkscited by 2
  9. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2
  10. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  11. 2025A Survey on Training-free Alignment of Large Language Modelscited by 2
  12. 2023On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQRcited by 1
6 more
  1. 2026Contextual Intelligence The Next Leap for Reinforcement Learningcited by 1
  2. 2025On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementationscited by 0
  3. 2025Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contextscited by 0
  4. 2025Adaptive Policy Backbone via Shared Networkcited by 0
  5. 2026Dynamic Context Scheduling: Learning Beyond the Static Universecited by 0
  6. 2026Amortized Low-Rank Adaptation for Model-Based Reinforcement Learningcited by 0