# NoRML: No-Reward Meta Learning

arXiv: https://arxiv.org/abs/1903.01063 · HTML: https://mlsanity.com/paper/1903.01063/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 7 of 7)

- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Offline Meta-Reinforcement Learning with Advantage Weighting](https://mlsanity.com/paper/2008.06043/) (2020)
- [Skill-based Meta-Reinforcement Learning](https://mlsanity.com/paper/2204.11828/) (2022)
- [Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills](https://mlsanity.com/paper/2312.06518/) (2023)
- [Unsupervised Meta-Testing with Conditional Neural Processes for Hybrid Meta-Reinforcement Learning](https://mlsanity.com/paper/2506.04399/) (2025)
- [Meta-World Conditional Neural Processes](https://mlsanity.com/paper/2302.10320/) (2023)
- [Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments](https://mlsanity.com/paper/2101.07599/) (2021)

## Abstract (arXiv)

Efficiently adapting to new environments and changes in dynamics is critical for agents to successfully operate in the real world. Reinforcement learning (RL) based approaches typically rely on external reward feedback for adaptation. However, in many scenarios this reward signal might not be readily available for the target task, or the difference between the environments can be implicit and only observable from the dynamics. To this end, we introduce a method that allows for self-adaptation of learned policies: No-Reward Meta Learning (NoRML). NoRML extends Model Agnostic Meta Learning (MAML) for RL and uses observable dynamics of the environment instead of an explicit reward function in MAML's finetune step. Our method has a more expressive update step than MAML, while maintaining MAML's gradient based foundation. Additionally, in order to allow more targeted exploration, we implement an extension to MAML that effectively disconnects the meta-policy parameters from the fine-tuned policies' parameters. We first study our method on a number of synthetic control problems and then validate our method on common benchmark environments, showing that NoRML outperforms MAML when the dynamics change between tasks.