mlsanityData to 14 Oct 2026

Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning

arXiv 2609.12278 ↗First posted 10 Sep 2026cs.LG cs.AI cs.ROcited by 0

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

World models let agents plan by predicting the consequences of their actions, but changes in the environment can make them inaccurate. We study the problem of adapting a world model to an unknown test-time environment, drawn from a known environment family, using only a few episodes of interaction. Existing approaches trade off computational cost against expressivity, i.e., the range of models a method can produce. For example, in-context learning is computationally cheap but limited in expressivity, and gradient-based adaptation is expressive but computationally expensive. We present CLAW (Context-conditioned Low-rank Adaptation of World models), which addresses this tradeoff by using a hypernetwork to generate low-rank (LoRA) adapters at test time. During pretraining, we simulate adaptation to a variety of environments and jointly train the hypernetwork and base world model. At test time, we freeze the base model and use a forward pass of the hypernetwork to generate adapters from a small batch of test-time transitions. We evaluate CLAW in locomotion and manipulation environment families that vary in dynamics, embodiment, and reward. We show that, using only seconds of test-time data, CLAW outperforms gradient-based adaptation and in-context learning during online adaptation. We also show that CLAW avoids overfitting in data-scarce regimes, that its advantage comes from the expressive adapters rather than context conditioning, and that pretraining the hypernetwork jointly with the base model outperforms training it post hoc.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 62 of its 70 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

62 of 70 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Adapting World Models with Latent-State Dynamics Residuals3 Apr
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  3. 2023Mastering Diverse Domains through World Models10 Jan
  4. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning16 Aug
  5. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  6. 2018World Models27 Mar

Other references · 56

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2016HyperNetworkscited by 1,027
  9. 2018DeepMind Control Suitecited by 956
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  11. 2022A Generalist Agentcited by 808
  12. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
38 more
  1. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  2. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  3. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  4. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  5. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  6. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  7. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  8. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  9. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  10. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 284
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  12. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  13. 2019Continual learning with hypernetworkscited by 273
  14. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 273
  15. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  16. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  17. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  18. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 124
  19. 2022Tutorial on amortized optimizationcited by 83
  20. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  21. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 66
  22. 2025Text-to-LoRA: Instant Transformer Adaptioncited by 58
  23. 2018Meta-Learning Priors for Efficient Online Bayesian Regressioncited by 48
  24. 2020Continual Model-Based Reinforcement Learning with Hypernetworkscited by 44
  25. 2022Hypernetworks in Meta-Reinforcement Learningcited by 42
  26. 2020ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representationscited by 38
  27. 2021Learning to Generate Task-Specific Adapters from Task Descriptioncited by 28
  28. 2024Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Passcited by 26
  29. 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Passcited by 25
  30. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  31. 2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 20
  32. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 19
  33. 2021HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworkscited by 18
  34. 2022HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworkscited by 17
  35. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuningcited by 17
  36. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  37. 2025Latent Policy Steering with Embodiment-Agnostic Pretrained World Modelscited by 8
  38. 2025Neural Robot Dynamicscited by 8

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents
  2. 2026Normalized Low-Rank Adaptation
  3. 2025DiWA: Diffusion Policy Adaptation with World Models
  4. 2025Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning
  5. 2026Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models
  6. 2026Inference Time Policy Optimization for Offline RL with Differentiable World Models
  7. 2024Partial Models for Building Adaptive Model-Based Reinforcement Learning Agents
  8. 2026Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning