mlsanityData to 7 Oct 2026

Separation-Utility Pareto Frontier: An Information-Theoretic Characterization

arXiv 2602.04408 ↗First posted 4 Feb 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study the Pareto frontier (optimal trade-off) between utility and separation, a fairness criterion requiring predictive independence from sensitive attributes conditional on the true outcome. Through an information-theoretic lens, we prove a characterization of the utility-separation Pareto frontier, establish its concavity, and thereby prove the increasing marginal cost of separation in terms of utility. In addition, we characterize the conditions under which this trade-off becomes strict, providing a guide for trade-off selection in practice. Based on the theoretical characterization, we develop an empirical regularizer based on conditional mutual information (CMI) between predictions and sensitive attributes given the true outcome. The CMI regularizer is compatible with any deep model trained via gradient-based optimization and serves as a scalar monitor of residual separation violations, offering tractable guarantees during training. Finally, numerical experiments support our theoretical findings: across COMPAS, UCI Adult, UCI Bank, and CelebA, the proposed method substantially reduces separation violations while matching or exceeding the utility of established baseline methods. This study thus offers a provable, stable, and flexible approach to enforcing separation in deep learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

25 of 41 references.

References · 25

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2018MINE: Mutual Information Neural Estimationcited by 680
  6. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  7. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2018A Reductions Approach to Fair Classificationcited by 585
  10. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2017On Fairness and Calibrationcited by 406
13 more
  1. 2017Algorithmic decision making and the cost of fairnesscited by 402
  2. 2018Learning Adversarially Fair and Transferable Representationscited by 353
  3. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
  4. 2018Classification with Fairness Constraints: A Meta-Algorithm with Provable Guaranteescited by 146
  5. 2019Understanding the Limitations of Variational Mutual Information Estimatorscited by 140
  6. 2018Learning Controllable Fair Representationscited by 83
  7. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  8. 2018Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraintscited by 53
  9. 2020FR-Train: A Mutual Information-Based Approach to Fair and Robust Trainingcited by 52
  10. 2020Achieving Equalized Odds by Resampling Sensitive Attributescited by 42
  11. 2022Fair Data Representation for Machine Learning at the Pareto Frontiercited by 12
  12. 2020Fast Fair Regression via Efficient Approximations of Mutual Informationcited by 10
  13. 2025Machine Unlearning via Information Theoretic Regularizationcited by 5

Built on by

Not cited yet.