# Kernel Dependence Regularizers and Gaussian Processes with Applications to Algorithmic Fairness

arXiv: https://arxiv.org/abs/1911.04322 · HTML: https://mlsanity.com/paper/1911.04322/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 16 of 16)

- [FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods](https://mlsanity.com/paper/2306.09468/) (2023)
- [RKHS-SHAP: Shapley Values for Kernel Methods](https://mlsanity.com/paper/2110.09167/) (2021)
- [Non-Gaussian Gaussian Processes for Few-Shot Regression](https://mlsanity.com/paper/2110.13561/) (2021)
- [Towards Learning an Unbiased Classifier from Biased Data via Conditional Adversarial Debiasing](https://mlsanity.com/paper/2103.06179/) (2021)
- [Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in ML](https://mlsanity.com/paper/2411.11101/) (2024)
- [Enabling Group Fairness in Graph Unlearning via Bi-level Debiasing](https://mlsanity.com/paper/2505.09702/) (2025)
- [Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective](https://mlsanity.com/paper/2506.07861/) (2025)
- [A statistical approach to detect sensitive features in a group fairness setting](https://mlsanity.com/paper/2305.06994/) (2023)
- [HARD: Hard Augmentations for Robust Distillation](https://mlsanity.com/paper/2305.14890/) (2023)
- [AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescales](https://mlsanity.com/paper/2607.05100/) (2026)
- [Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge](https://mlsanity.com/paper/2301.11214/) (2023)
- [Out-of-distribution robustness for multivariate analysis via causal regularisation](https://mlsanity.com/paper/2403.01865/) (2024)
- [Advancing Fairness in Natural Language Processing: From Traditional Methods to Explainability](https://mlsanity.com/paper/2410.12511/) (2024)
- [Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networks](https://mlsanity.com/paper/2507.04033/) (2025)
- [Geometry of Relaxed Fair Regression: A Unified Framework for Aware and Unaware Settings](https://mlsanity.com/paper/2605.28233/) (2026)
- [A Joint-Distribution Route to Fair Representations with Continuous Sensitive Attributes](https://mlsanity.com/paper/2608.10470/) (2026)

## Abstract (arXiv)

Current adoption of machine learning in industrial, societal and economical activities has raised concerns about the fairness, equity and ethics of automated decisions. Predictive models are often developed using biased datasets and thus retain or even exacerbate biases in their decisions and recommendations. Removing the sensitive covariates, such as gender or race, is insufficient to remedy this issue since the biases may be retained due to other related covariates. We present a regularization approach to this problem that trades off predictive accuracy of the learned models (with respect to biased labels) for the fairness in terms of statistical parity, i.e. independence of the decisions from the sensitive covariates. In particular, we consider a general framework of regularized empirical risk minimization over reproducing kernel Hilbert spaces and impose an additional regularizer of dependence between predictors and sensitive covariates using kernel-based measures of dependence, namely the Hilbert-Schmidt Independence Criterion (HSIC) and its normalized version. This approach leads to a closed-form solution in the case of squared loss, i.e. ridge regression. Moreover, we show that the dependence regularizer has an interpretation as modifying the corresponding Gaussian process (GP) prior. As a consequence, a GP model with a prior that encourages fairness to sensitive variables can be derived, allowing principled hyperparameter selection and studying of the relative relevance of covariates under fairness constraints. Experimental results in synthetic examples and in real problems of income and crime prediction illustrate the potential of the approach to improve fairness of automated decisions.