# Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning

arXiv: https://arxiv.org/abs/2110.03909 · HTML: https://mlsanity.com/paper/2110.03909/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 67 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 30 of 48)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/1805.09801/) (2018)
- [Task-Agnostic Meta-Learning for Few-shot Learning](https://mlsanity.com/paper/1805.07722/) (2018)
- [Interventional Few-Shot Learning](https://mlsanity.com/paper/2009.13000/) (2020)
- [Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation](https://mlsanity.com/paper/1910.13616/) (2019)
- [Meta-Learning with Warped Gradient Descent](https://mlsanity.com/paper/1909.00025/) (2019)

## Built on by (most cited first, 30 of 49)

- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning](https://mlsanity.com/paper/2307.16424/) (2023)
- [Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning](https://mlsanity.com/paper/2203.09064/) (2022)
- [Task-driven Image Fusion with Learnable Fusion Loss](https://mlsanity.com/paper/2412.03240/) (2024)
- [Supervised Masked Knowledge Distillation for Few-Shot Transformers](https://mlsanity.com/paper/2303.15466/) (2023)
- [Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid](https://mlsanity.com/paper/2210.16834/) (2022)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learning](https://mlsanity.com/paper/2312.07943/) (2023)
- [CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections](https://mlsanity.com/paper/2403.03063/) (2024)
- [Meta-Learning with a Geometry-Adaptive Preconditioner](https://mlsanity.com/paper/2304.01552/) (2023)
- [LFS-GAN: Lifelong Few-Shot Image Generation](https://mlsanity.com/paper/2308.11917/) (2023)
- [Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification](https://mlsanity.com/paper/2209.08527/) (2022)
- [Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning](https://mlsanity.com/paper/2209.08907/) (2022)
- [Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile](https://mlsanity.com/paper/2206.01944/) (2022)
- [Sample-Level Weighting for Multi-Task Learning with Auxiliary Tasks](https://mlsanity.com/paper/2306.04519/) (2023)
- [Few-Shot Learning of Compact Models via Task-Specific Meta Distillation](https://mlsanity.com/paper/2210.09922/) (2022)
- [Meta-Learning Adaptive Loss Functions](https://mlsanity.com/paper/2301.13247/) (2023)
- [Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs](https://mlsanity.com/paper/2302.03235/) (2023)
- [Meta-DM: Applications of Diffusion Models on Few-Shot Learning](https://mlsanity.com/paper/2305.08092/) (2023)
- [Curvature Learning for Generalization of Hyperbolic Neural Networks](https://mlsanity.com/paper/2508.17232/) (2025)
- [Improve Noise Tolerance of Robust Loss via Noise-Awareness](https://mlsanity.com/paper/2301.07306/) (2023)
- [Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation](https://mlsanity.com/paper/2302.14640/) (2023)
- [Few-shot Classification via Ensemble Learning with Multi-Order Statistics](https://mlsanity.com/paper/2305.00454/) (2023)
- [Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Environmental Systems](https://mlsanity.com/paper/2310.04727/) (2023)
- [MetaDT: Meta Decision Tree with Class Hierarchy for Interpretable Few-Shot Learning](https://mlsanity.com/paper/2203.01482/) (2022)
- [Meta-Auxiliary Network for 3D GAN Inversion](https://mlsanity.com/paper/2305.10884/) (2023)
- [Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworks](https://mlsanity.com/paper/2307.06771/) (2023)
- [L2T-DLN: Learning to Teach with Dynamic Loss Network](https://mlsanity.com/paper/2310.19313/) (2023)
- [AdaCCD: Adaptive Semantic Contrasts Discovery Based Cross Lingual Adaptation for Code Clone Detection](https://mlsanity.com/paper/2311.07277/) (2023)
- [Meta-Learning with Heterogeneous Tasks](https://mlsanity.com/paper/2410.18894/) (2024)

## Abstract (arXiv)

In few-shot learning scenarios, the challenge is to generalize and perform well on new unseen examples when only very few labeled examples are available for each task. Model-agnostic meta-learning (MAML) has gained the popularity as one of the representative few-shot learning methods for its flexibility and applicability to diverse problems. However, MAML and its variants often resort to a simple loss function without any auxiliary loss function or regularization terms that can help achieve better generalization. The problem lies in that each application and task may require different auxiliary loss function, especially when tasks are diverse and distinct. Instead of attempting to hand-design an auxiliary loss function for each application and task, we introduce a new meta-learning framework with a loss function that adapts to each task. Our proposed framework, named Meta-Learning with Task-Adaptive Loss Function (MeTAL), demonstrates the effectiveness and the flexibility across various domains, such as few-shot classification and few-shot regression.