# Meta-Learning via Learned Loss

arXiv: https://arxiv.org/abs/1906.05374 · HTML: https://mlsanity.com/paper/1906.05374/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2019.

## Built on by (most cited first, 30 of 60)

- [General-Purpose In-Context Learning by Meta-Learning Transformers](https://mlsanity.com/paper/2212.04458/) (2022)
- [Test-Time Adaptation via Conjugate Pseudo-labels](https://mlsanity.com/paper/2207.09640/) (2022)
- [Offline Meta-Reinforcement Learning with Advantage Weighting](https://mlsanity.com/paper/2008.06043/) (2020)
- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [Hyperparameters in Reinforcement Learning and How To Tune Them](https://mlsanity.com/paper/2306.01324/) (2023)
- [Model-Based Inverse Reinforcement Learning from Visual Demonstrations](https://mlsanity.com/paper/2010.09034/) (2020)
- [Meta-learning PINN loss functions](https://mlsanity.com/paper/2107.05544/) (2021)
- [Evolving Reinforcement Learning Algorithms](https://mlsanity.com/paper/2101.03958/) (2021)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks](https://mlsanity.com/paper/2410.23208/) (2024)
- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)
- [Prompt Learning via Meta-Regularization](https://mlsanity.com/paper/2404.00851/) (2024)
- [Introducing Symmetries to Black Box Meta Reinforcement Learning](https://mlsanity.com/paper/2109.10781/) (2021)
- [Megaverse: Simulating Embodied Agents at One Million Experiences per Second](https://mlsanity.com/paper/2107.08170/) (2021)
- [Learning Pessimism for Robust and Efficient Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2110.03375/) (2021)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [Discovering Temporally-Aware Reinforcement Learning Algorithms](https://mlsanity.com/paper/2402.05828/) (2024)
- [Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design](https://mlsanity.com/paper/2310.02782/) (2023)
- [Neural Relational Inference with Fast Modular Meta-learning](https://mlsanity.com/paper/2310.07015/) (2023)
- [Behavior Alignment via Reward Function Optimization](https://mlsanity.com/paper/2310.19007/) (2023)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [Searching for Robustness: Loss Learning for Noisy Classification Tasks](https://mlsanity.com/paper/2103.00243/) (2021)
- [A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning](https://mlsanity.com/paper/2112.15400/) (2021)
- [Learning to Optimize for Reinforcement Learning](https://mlsanity.com/paper/2302.01470/) (2023)
- [Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps](https://mlsanity.com/paper/2412.17113/) (2024)
- [Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning](https://mlsanity.com/paper/2209.08907/) (2022)
- [BC-IRL: Learning Generalizable Reward Functions from Demonstrations](https://mlsanity.com/paper/2303.16194/) (2023)
- [One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/2111.00206/) (2021)
- [Meta-Learning Adaptive Loss Functions](https://mlsanity.com/paper/2301.13247/) (2023)

## Abstract (arXiv)

Typically, loss functions, regularization mechanisms and other important aspects of training parametric models are chosen heuristically from a limited set of options. In this paper, we take the first step towards automating this process, with the view of producing models which train faster and more robustly. Concretely, we present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for meta-training such loss functions, targeted at maximizing the performance of the model trained under them. The loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional information at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time. We make our code available at https://sites.google.com/view/mlthree.