mlsanityData to 7 Oct 2026

Metadata Normalization

arXiv 2104.09052 ↗First posted 19 Apr 2021cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Batch Normalization (BN) and its variants have delivered tremendous success in combating the covariate shift induced by the training step of deep learning methods. While these techniques normalize feature distributions by standardizing with batch statistics, they do not correct the influence on features from extraneous variables or multiple distributions. Such extra variables, referred to as metadata here, may create bias or confounding effects (e.g., race when classifying gender from face images). We introduce the Metadata Normalization (MDN) layer, a new batch-level operation which can be used end-to-end within the training framework, to correct the influence of metadata on feature distributions. MDN adopts a regression analysis technique traditionally used for preprocessing to remove (regress out) the metadata effects on model features during training. We utilize a metric based on distance correlation to quantify the distribution bias from the metadata and demonstrate that our method successfully removes metadata effects on four diverse settings: one synthetic, one 2D image, one video, and one 3D medical image dataset.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 30 of its 71 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

30 of 71 references.

References · 30

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Layer Normalizationcited by 5,579
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  6. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  7. 2018Group Normalizationcited by 1,514
  8. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  9. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  10. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  11. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  12. 2018Learning Adversarially Fair and Transferable Representationscited by 353
18 more
  1. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  2. 2015The Variational Fair Autoencodercited by 223
  3. 2017Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognitioncited by 178
  4. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  5. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 152
  6. 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
  7. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  8. 2019Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognitioncited by 117
  9. 2018Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddingscited by 114
  10. 2018Invariant Representations without Adversarial Trainingcited by 111
  11. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
  12. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
  13. 2020Detect and Correct Bias in Multi-Site Neuroimaging Datasetscited by 44
  14. 2019Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalizationcited by 43
  15. 2019Learning Fair Representations for Kernel Modelscited by 17
  16. 2019Confounder-Aware Visualization of ConvNetscited by 7
  17. 2017AdaBatch: Efficient Gradient Aggregation Rules for Sequential and Parallel Stochastic Gradient Methodscited by 5
  18. 2018Removing Confounding Factors Associated Weights in Deep Neural Networks Improves the Prediction Accuracy for Healthcare Applicationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 3; 2025: 1; 2026: 2.

14 papers.

Sort:

  1. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  2. 2023Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIscited by 12
  3. 2023Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeatcited by 10
  4. 2022A Penalty Approach for Normalizing Feature Distributions to Build Confounder-Free Modelscited by 8
  5. 2024DiaMond: Dementia Diagnosis with Multi-Modal Vision Transformers Using MRI and PETcited by 6
  6. 2023RegBN: Batch Normalization of Multimodal Data with Regularizationcited by 5
  7. 2023Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorizationcited by 4
  8. 2024Enforcing Conditional Independence for Fair Representation Learning and Causal Image Generationcited by 4
  9. 2023Tackling Shortcut Learning in Deep Neural Networks: An Iterative Approach with Interpretable Modelscited by 2
  10. 2023AI-based association analysis for medical imaging using latent-space geometric confounder correctioncited by 1
  11. 2024Generalizing Orthogonalization for Models with Non-Linearitiescited by 0
  12. 2025Confounder-Free Continual Learning via Recursive Feature Normalizationcited by 0
2 more
  1. 2026Interpreting Protein Language Model Embeddings via Orthogonal Projection for Protein Fitness Predictioncited by 0
  2. 2026Controlling for Omitted Variable Bias in Deep Neural Networkscited by 0