# Conditional Meta-Learning of Linear Representations

arXiv: https://arxiv.org/abs/2103.16277 · HTML: https://mlsanity.com/paper/2103.16277/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Adaptive Gradient-Based Meta-Learning Methods](https://mlsanity.com/paper/1906.02717/) (2019)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [A PAC-Bayesian bound for Lifelong Learning](https://mlsanity.com/paper/1311.2838/) (2013)
- [Hierarchically Structured Meta-learning](https://mlsanity.com/paper/1905.05301/) (2019)
- [Provable Guarantees for Gradient-Based Meta-Learning](https://mlsanity.com/paper/1902.10644/) (2019)
- [Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm](https://mlsanity.com/paper/1710.11622/) (2017)
- [Learning-to-Learn Stochastic Gradient Descent with Biased Regularization](https://mlsanity.com/paper/1903.10399/) (2019)
- [The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning](https://mlsanity.com/paper/2008.10857/) (2020)
- [Incremental Learning-to-Learn with Statistical Guarantees](https://mlsanity.com/paper/1803.08089/) (2018)
- [New Perspectives on k-Support and Cluster Norms](https://mlsanity.com/paper/1512.08204/) (2015)
- [Weighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information](https://mlsanity.com/paper/1709.03907/) (2017)
- [Structured Prediction for Conditional Meta-Learning](https://mlsanity.com/paper/2002.08799/) (2020)

## Built on by (most cited first, 12 of 12)

- [How Fine-Tuning Allows for Effective Meta-Learning](https://mlsanity.com/paper/2105.02221/) (2021)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [Provable Lifelong Learning of Representations](https://mlsanity.com/paper/2110.14098/) (2021)
- [Learning useful representations for shifting tasks and distributions](https://mlsanity.com/paper/2212.07346/) (2022)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [Meta-Learning with Adjoint Methods](https://mlsanity.com/paper/2110.08432/) (2021)
- [Entity Aware Modelling: A Survey](https://mlsanity.com/paper/2302.08406/) (2023)
- [Robust Meta-Representation Learning via Global Label Inference and Classification](https://mlsanity.com/paper/2212.11702/) (2022)
- [Towards Automated Knowledge Integration From Human-Interpretable Representations](https://mlsanity.com/paper/2402.16105/) (2024)
- [Meta-Learning Linear Models for Molecular Property Prediction](https://mlsanity.com/paper/2509.13527/) (2025)
- [Meta-learning Amidst Heterogeneity and Ambiguity](https://mlsanity.com/paper/2107.02228/) (2021)
- [Capacity and Redundancy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2607.16554/) (2026)

## Abstract (arXiv)

Standard meta-learning for representation learning aims to find a common representation to be shared across multiple tasks. The effectiveness of these methods is often limited when the nuances of the tasks' distribution cannot be captured by a single representation. In this work we overcome this issue by inferring a conditioning function, mapping the tasks' side information (such as the tasks' training dataset itself) into a representation tailored to the task at hand. We study environments in which our conditional strategy outperforms standard meta-learning, such as those in which tasks can be organized in separate clusters according to the representation they share. We then propose a meta-algorithm capable of leveraging this advantage in practice. In the unconditional setting, our method yields a new estimator enjoying faster learning rates and requiring less hyper-parameters to tune than current state-of-the-art methods. Our results are supported by preliminary experiments.