# A Penalty Approach for Normalizing Feature Distributions to Build Confounder-Free Models

arXiv: https://arxiv.org/abs/2207.04607 · HTML: https://mlsanity.com/paper/2207.04607/

## Key facts

- Cited by 8 papers in the map (data through 7 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning](https://mlsanity.com/paper/1611.08331/) (2016)
- [Metadata Normalization](https://mlsanity.com/paper/2104.09052/) (2021)
- [Projection-wise Disentangling for Fair and Interpretable Representation Learning: Application to 3D Facial Shape Analysis](https://mlsanity.com/paper/2106.13734/) (2021)
- [Combining Counterfactuals With Shapley Values To Explain Image Models](https://mlsanity.com/paper/2206.07087/) (2022)
- [Bridging the Generalization Gap: Training Robust Models on Confounded Biological Data](https://mlsanity.com/paper/1812.04778/) (2018)

## Built on by (most cited first, 8 of 8)

- [Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Review](https://mlsanity.com/paper/2209.13177/) (2022)
- [RegBN: Batch Normalization of Multimodal Data with Regularization](https://mlsanity.com/paper/2310.00641/) (2023)
- [Enforcing Conditional Independence for Fair Representation Learning and Causal Image Generation](https://mlsanity.com/paper/2404.13798/) (2024)
- [FairREAD: Re-fusing Demographic Attributes after Disentanglement for Fair Medical Image Classification](https://mlsanity.com/paper/2412.16373/) (2024)
- [AI-based association analysis for medical imaging using latent-space geometric confounder correction](https://mlsanity.com/paper/2311.12836/) (2023)
- [Generalizing Orthogonalization for Models with Non-Linearities](https://mlsanity.com/paper/2405.02475/) (2024)
- [Confounder-Free Continual Learning via Recursive Feature Normalization](https://mlsanity.com/paper/2507.09031/) (2025)
- [Controlling for Omitted Variable Bias in Deep Neural Networks](https://mlsanity.com/paper/2608.25930/) (2026)

## Abstract (arXiv)

Translating machine learning algorithms into clinical applications requires addressing challenges related to interpretability, such as accounting for the effect of confounding variables (or metadata). Confounding variables affect the relationship between input training data and target outputs. When we train a model on such data, confounding variables will bias the distribution of the learned features. A recent promising solution, MetaData Normalization (MDN), estimates the linear relationship between the metadata and each feature based on a non-trainable closed-form solution. However, this estimation is confined by the sample size of a mini-batch and thereby may cause the approach to be unstable during training. In this paper, we extend the MDN method by applying a Penalty approach (referred to as PDMN). We cast the problem into a bi-level nested optimization problem. We then approximate this optimization problem using a penalty method so that the linear parameters within the MDN layer are trainable and learned on all samples. This enables PMDN to be plugged into any architectures, even those unfit to run batch-level operations, such as transformers and recurrent models. We show improvement in model accuracy and greater independence from confounders using PMDN over MDN in a synthetic experiment and a multi-label, multi-site dataset of magnetic resonance images (MRIs).