# The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning

arXiv: https://arxiv.org/abs/2008.10857 · HTML: https://mlsanity.com/paper/2008.10857/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 30 of 40)

- [Advances and Challenges in Meta-Learning: A Technical Review](https://mlsanity.com/paper/2307.04722/) (2023)
- [How Fine-Tuning Allows for Effective Meta-Learning](https://mlsanity.com/paper/2105.02221/) (2021)
- [Meta Knowledge Condensation for Federated Learning](https://mlsanity.com/paper/2209.14851/) (2022)
- [CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning](https://mlsanity.com/paper/2202.05613/) (2022)
- [Distribution Regression with Sliced Wasserstein Kernels](https://mlsanity.com/paper/2202.03926/) (2022)
- [Provable Lifelong Learning of Representations](https://mlsanity.com/paper/2110.14098/) (2021)
- [Towards Constituting Mathematical Structures for Learning to Optimize](https://mlsanity.com/paper/2305.18577/) (2023)
- [Conditional Meta-Learning of Linear Representations](https://mlsanity.com/paper/2103.16277/) (2021)
- [The Role of Global Labels in Few-Shot Classification and How to Infer Them](https://mlsanity.com/paper/2108.04055/) (2021)
- [An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning](https://mlsanity.com/paper/2101.08390/) (2021)
- [The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence](https://mlsanity.com/paper/2208.01545/) (2022)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback](https://mlsanity.com/paper/2107.14035/) (2021)
- [Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes](https://mlsanity.com/paper/2302.11709/) (2023)
- [Task-Adaptive Meta-Learning Framework for Advancing Spatial Generalizability](https://mlsanity.com/paper/2212.06864/) (2022)
- [Meta-strategy for Learning Tuning Parameters with Guarantees](https://mlsanity.com/paper/2102.02504/) (2021)
- [Is Pre-training Truly Better Than Meta-Learning?](https://mlsanity.com/paper/2306.13841/) (2023)
- [Meta-Learning with Adjoint Methods](https://mlsanity.com/paper/2110.08432/) (2021)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stability](https://mlsanity.com/paper/2305.19694/) (2023)
- [Robust Meta-Representation Learning via Global Label Inference and Classification](https://mlsanity.com/paper/2212.11702/) (2022)
- [Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification](https://mlsanity.com/paper/2312.07338/) (2023)
- [Towards Automated Knowledge Integration From Human-Interpretable Representations](https://mlsanity.com/paper/2402.16105/) (2024)
- [Bridging Domains with Approximately Shared Features](https://mlsanity.com/paper/2403.06424/) (2024)
- [Fast and Scalable Score-Based Kernel Calibration Tests](https://mlsanity.com/paper/2510.14711/) (2025)
- [Learning Tensor Representations for Meta-Learning](https://mlsanity.com/paper/2201.07348/) (2022)
- [Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition](https://mlsanity.com/paper/2106.04392/) (2021)
- [Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adapter](https://mlsanity.com/paper/2309.09443/) (2023)
- [GMTR: Graph Matching Transformers](https://mlsanity.com/paper/2311.08141/) (2023)
- [DarkShot: Lighting Dark Images with Low-Compute and High-Quality](https://mlsanity.com/paper/2312.16805/) (2023)

## Abstract (arXiv)

Biased regularization and fine-tuning are two recent meta-learning approaches. They have been shown to be effective to tackle distributions of tasks, in which the tasks' target vectors are all close to a common meta-parameter vector. However, these methods may perform poorly on heterogeneous environments of tasks, where the complexity of the tasks' distribution cannot be captured by a single meta-parameter vector. We address this limitation by conditional meta-learning, inferring a conditioning function mapping task's side information into a meta-parameter vector that is appropriate for that task at hand. We characterize properties of the environment under which the conditional approach brings a substantial advantage over standard meta-learning and we highlight examples of environments, such as those with multiple clusters, satisfying these properties. We then propose a convex meta-algorithm providing a comparable advantage also in practice. Numerical experiments confirm our theoretical findings.