# On the Global Optima of Kernelized Adversarial Representation Learning

arXiv: https://arxiv.org/abs/1910.07423 · HTML: https://mlsanity.com/paper/1910.07423/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2019.

## Built on by (most cited first, 14 of 14)

- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [Kernelized Concept Erasure](https://mlsanity.com/paper/2201.12191/) (2022)
- [Utility-Fairness Trade-Offs and How to Find Them](https://mlsanity.com/paper/2404.09454/) (2024)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [On Characterizing the Trade-off in Invariant Representation Learning](https://mlsanity.com/paper/2109.03386/) (2021)
- [Adversarial Representation Learning With Closed-Form Solvers](https://mlsanity.com/paper/2109.05535/) (2021)
- [Privacy Safe Representation Learning via Frequency Filtering Encoder](https://mlsanity.com/paper/2208.02482/) (2022)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Confounder-Free Continual Learning via Recursive Feature Normalization](https://mlsanity.com/paper/2507.09031/) (2025)
- [Learning Private Representations through Entropy-based Adversarial Training](https://mlsanity.com/paper/2507.10194/) (2025)
- [Private Vertical Federated Inference for Time-Series](https://mlsanity.com/paper/2605.08343/) (2026)
- [Controlling for Omitted Variable Bias in Deep Neural Networks](https://mlsanity.com/paper/2608.25930/) (2026)

## Abstract (arXiv)

Adversarial representation learning is a promising paradigm for obtaining data representations that are invariant to certain sensitive attributes while retaining the information necessary for predicting target attributes. Existing approaches solve this problem through iterative adversarial minimax optimization and lack theoretical guarantees. In this paper, we first study the "linear" form of this problem i.e., the setting where all the players are linear functions. We show that the resulting optimization problem is both non-convex and non-differentiable. We obtain an exact closed-form expression for its global optima through spectral learning and provide performance guarantees in terms of analytical bounds on the achievable utility and invariance. We then extend this solution and analysis to non-linear functions through kernel representation. Numerical experiments on UCI, Extended Yale B and CIFAR-100 datasets indicate that, (a) practically, our solution is ideal for "imparting" provable invariance to any biased pre-trained data representation, and (b) empirically, the trade-off between utility and invariance provided by our solution is comparable to iterative minimax optimization of existing deep neural network based approaches. Code is available at https://github.com/human-analysis/Kernel-ARL