# Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks

arXiv: https://arxiv.org/abs/1905.12917 · HTML: https://mlsanity.com/paper/1905.12917/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 30 of 48)

- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models](https://mlsanity.com/paper/2210.14199/) (2022)
- [Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification](https://mlsanity.com/paper/2110.14012/) (2021)
- [Uncertainty-guided Model Generalization to Unseen Domains](https://mlsanity.com/paper/2103.07531/) (2021)
- [Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets](https://mlsanity.com/paper/2107.00860/) (2021)
- [Realistic Evaluation of Transductive Few-Shot Learning](https://mlsanity.com/paper/2204.11181/) (2022)
- [Few-Shot Learning with Class Imbalance](https://mlsanity.com/paper/2101.02523/) (2021)
- [Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach](https://mlsanity.com/paper/2108.02888/) (2021)
- [HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning](https://mlsanity.com/paper/2106.08630/) (2021)
- [Exploring Efficient Few-shot Adaptation for Vision Transformers](https://mlsanity.com/paper/2301.02419/) (2023)
- [Meta-Learning with Neural Tangent Kernels](https://mlsanity.com/paper/2102.03909/) (2021)
- [Fine-tune Language Models to Approximate Unbiased In-context Learning](https://mlsanity.com/paper/2310.03331/) (2023)
- [PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior](https://mlsanity.com/paper/2102.03748/) (2021)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution](https://mlsanity.com/paper/2102.11503/) (2021)
- [Multidimensional Belief Quantification for Label-Efficient Meta-Learning](https://mlsanity.com/paper/2203.12768/) (2022)
- [Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2207.00784/) (2022)
- [Meta-learning in healthcare: A survey](https://mlsanity.com/paper/2308.02877/) (2023)
- [Task-Adaptive Neural Network Search with Meta-Contrastive Learning](https://mlsanity.com/paper/2103.01495/) (2021)
- [Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness](https://mlsanity.com/paper/2109.14120/) (2021)
- [Meta-Learned Invariant Risk Minimization](https://mlsanity.com/paper/2103.12947/) (2021)
- [Contextualizing Meta-Learning via Learning to Decompose](https://mlsanity.com/paper/2106.08112/) (2021)
- [RotoGBML: Towards Out-of-Distribution Generalization for Gradient-Based Meta-Learning](https://mlsanity.com/paper/2303.06679/) (2023)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks](https://mlsanity.com/paper/2205.15619/) (2022)
- [Task-Distributionally Robust Data-Free Meta-Learning](https://mlsanity.com/paper/2311.14756/) (2023)
- [Panoptic Perception for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.15388/) (2024)
- [Hand-Shadow Poser](https://mlsanity.com/paper/2505.07012/) (2025)
- [Distribution Embedding Networks for Generalization from a Diverse Set of Classification Tasks](https://mlsanity.com/paper/2202.01940/) (2022)

## Abstract (arXiv)

While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed. Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks, even when the number of instances per task and class largely varies. Moreover, they do not consider distributional difference in unseen tasks, on which the meta-knowledge may have less usefulness depending on the task relatedness. To overcome these limitations, we propose a novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning within each task. Through the learning of the balancing variables, we can decide whether to obtain a solution by relying on the meta-knowledge or task-specific learning. We formulate this objective into a Bayesian inference framework and tackle it using variational inference. We validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on multiple realistic task- and class-imbalanced datasets, on which it significantly outperforms existing meta-learning approaches. Further ablation study confirms the effectiveness of each balancing component and the Bayesian learning framework.