mlsanityData to 7 Oct 2026

TaskNorm: Rethinking Batch Normalization for Meta-Learning

arXiv 2003.03284 ↗First posted 6 Mar 2020stat.ML cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

Modern meta-learning approaches for image classification rely on increasingly deep networks to achieve state-of-the-art performance, making batch normalization an essential component of meta-learning pipelines. However, the hierarchical nature of the meta-learning setting presents several challenges that can render conventional batch normalization ineffective, giving rise to the need to rethink normalization in this setting. We evaluate a range of approaches to batch normalization for meta-learning scenarios, and develop a novel approach that we call TaskNorm. Experiments on fourteen datasets demonstrate that the choice of batch normalization has a dramatic effect on both classification accuracy and training time for both gradient based and gradient-free meta-learning approaches. Importantly, TaskNorm is found to consistently improve performance. Finally, we provide a set of best practices for normalization that will allow fair comparison of meta-learning algorithms.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 16; 2022: 17; 2023: 8; 2024: 7; 2025: 2.

50 papers.

Sort:

By area:

  1. 2023TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptationcited by 101
  2. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  3. 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
  4. 2023A Closer Look at Few-shot Classification Againcited by 46
  5. 2022FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Drivingcited by 41
  6. 2022Continual Normalization: Rethinking Batch Normalization for Online Continual Learningcited by 40
  7. 2022Adversarial Feature Augmentation for Cross-domain Few-shot Classificationcited by 36
  8. 2021Learning where to learn: Gradient sparsity in meta and continual learningcited by 33
  9. 2023Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalizationcited by 31
  10. 2021Few-shot Open-set Recognition by Transformation Consistencycited by 29
  11. 2022Realistic Evaluation of Transductive Few-Shot Learningcited by 26
  12. 2024Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learningcited by 26
38 more
  1. 2024Learning to Defer to a Population: A Meta-Learning Approachcited by 26
  2. 2021Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learningcited by 24
  3. 2023DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecastingcited by 21
  4. 2021Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmarkcited by 16
  5. 2022Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learningcited by 16
  6. 2022Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classificationcited by 15
  7. 2022Evolution of Neural Tangent Kernels under Benign and Adversarial Trainingcited by 14
  8. 2023ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentationcited by 14
  9. 2024UniZero: Generalized and Efficient Planning with Scalable Latent World Modelscited by 12
  10. 2021What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Datacited by 10
  11. 2021Bridging Few-Shot Learning and Adaptation: New Challenges of Support-Query Shiftcited by 9
  12. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  13. 2022Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inferencecited by 9
  14. 2022Multi-Task Meta Learning: learn how to adapt to unseen taskscited by 8
  15. 2022Delving into the Estimation Shift of Batch Normalization in a Networkcited by 7
  16. 2021MetaMedSeg: Volumetric Meta-learning for Few-Shot Organ Segmentationcited by 6
  17. 2021Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta reviewcited by 6
  18. 2022Few-shot Open-set Recognition Using Background as Unknownscited by 6
  19. 2022Blind Users Accessing Their Training Images in Teachable Object Recognizerscited by 6
  20. 2023Metalearning Continual Learning Algorithmscited by 6
  21. 2021Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learningcited by 5
  22. 2021Transductive Few-Shot Learning: Clustering is All You Need?cited by 4
  23. 2022Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learningcited by 4
  24. 2022Diagnosing Batch Normalization in Class Incremental Learningcited by 4
  25. 2024A Dataset for Crucial Object Recognition in Blind and Low-Vision Individuals' Navigationcited by 4
  26. 2021Reciprocal Normalization for Domain Adaptationcited by 3
  27. 2023Support-Set Context Matters for Bongard Problemscited by 3
  28. 2023A transductive few-shot learning approach for classification of digital histopathological slides from liver cancercited by 3
  29. 2024Robust Few-Shot Ensemble Learning with Focal Diversity-Based Pruningcited by 3
  30. 2024Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentationcited by 2
  31. 2024CCSI: Continual Class-Specific Impression for Data-free Class Incremental Learningcited by 2
  32. 2022Semantics-driven Attentive Few-shot Learning over Clean and Noisy Samplescited by 1
  33. 2022Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reductioncited by 1
  34. 2025Feature-Based Instance Neighbor Discovery: Advanced Stable Test-Time Adaptation in Dynamic Worldcited by 1
  35. 2025Simplifying Multi-Task Architectures Through Task-Specific Normalizationcited by 1
  36. 2021Meta-Learning Conjugate Priors for Few-Shot Bayesian Optimizationcited by 0
  37. 2021Generating meta-learning tasks to evolve parametric loss for classification learningcited by 0
  38. 2022A Maximum Log-Likelihood Method for Imbalanced Few-Shot Learning Taskscited by 0