mlsanityData to 7 Oct 2026

Invariant Representations without Adversarial Training

arXiv 1805.09458 ↗First posted 24 May 2018cs.LG stat.MLcited by 111

Not on the map. Why?

Abstract (arXiv)

Representations of data that are invariant to changes in specified factors are useful for a wide range of problems: removing potential biases in prediction problems, controlling the effects of covariates, and disentangling meaningful factors of variation. Unfortunately, learning representations that exhibit invariance to arbitrary nuisance factors yet remain useful for other tasks is challenging. Existing approaches cast the trade-off between task performance and invariance in an adversarial way, using an iterative minimax optimization. We show that adversarial training is unnecessary and sometimes counter-productive; we instead cast invariant representation learning as a single information-theoretic objective that can be directly optimized. We demonstrate that this approach matches or exceeds performance of state-of-the-art adversarial approaches for learning fair representations and for generative modeling with controllable transformations.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 27; 2023: 18; 2024: 12; 2025: 14; 2026: 10.

111 papers.

Sort:

By area:

  1. 2022Bias Mitigation for Machine Learning Classifiers: A Comprehensive Surveycited by 125
  2. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  3. 2021Controllable Guarantees for Fair Outcomes via Contrastive Information Estimationcited by 45
  4. 2021Fair Normalizing Flowscited by 38
  5. 2021An Empirical Framework for Domain Generalization in Clinical Settingscited by 32
  6. 2021Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimizationcited by 32
  7. 2023Survey of Social Bias in Vision-Language Modelscited by 26
  8. 2021An Information-theoretic Approach to Distribution Shiftscited by 25
  9. 2023Efficient fair PCA for fair representation learningcited by 23
  10. 2021Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCEcited by 22
  11. 2022FARE: Provably Fair Representation Learning with Practical Certificatescited by 21
  12. 2021Embracing the Disharmony in Medical Imaging: A Simple and Effective Framework for Domain Adaptationcited by 19
38 more
  1. 2021Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputscited by 18
  2. 2022On the Versatile Uses of Partial Distance Correlation in Deep Learningcited by 18
  3. 2022Decoupling Local and Global Representations of Time Seriescited by 17
  4. 2021Fair Representation: Guaranteeing Approximate Multiple Group Fairness for Unknown Taskscited by 16
  5. 2021MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Datasetcited by 15
  6. 2021Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Explorationcited by 15
  7. 2021Metadata Normalizationcited by 14
  8. 2021Harmonization with Flow-based Causal Inferencecited by 14
  9. 2022A Contrastive Objective for Learning Disentangled Representationscited by 14
  10. 2022Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Makingcited by 12
  11. 2023FAIRER: Fairness as Decision Rationale Alignmentcited by 12
  12. 2022Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learningcited by 11
  13. 2021Fairness-Aware Unsupervised Feature Selectioncited by 10
  14. 2021Attributing Fair Decisions with Attention Interventionscited by 10
  15. 2022Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networkscited by 10
  16. 2023Toward Adversarial Training on Contextualized Language Representationcited by 10
  17. 2023Robust Concept Erasure via Kernelized Rate-Distortion Maximizationcited by 10
  18. 2024Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibriumcited by 9
  19. 2025Prediction-Powered Causal Inferencescited by 9
  20. 2021Combating Unknown Bias with Effective Bias-Conflicting Scoring and Gradient Alignmentcited by 8
  21. 2023Scalable Infomin Learningcited by 8
  22. 2024Disentangling representations of retinal images with generative modelscited by 8
  23. 2021Fair Representations by Compressioncited by 7
  24. 2022Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasetscited by 7
  25. 2023Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Toolscited by 7
  26. 2021Learning Invariant Representations using Inverse Contrastive Losscited by 6
  27. 2021VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Expertscited by 6
  28. 2021Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-losscited by 6
  29. 2021Controlling Text Edition by Changing Answers of Specific Questionscited by 6
  30. 2022Towards Group Robustness in the presence of Partial Group Labelscited by 6
  31. 2022Learning Fair Models without Sensitive Attributes: A Generative Approachcited by 6
  32. 2022Ultra-marginal Feature Importance: Learning from Data with Causal Guaranteescited by 6
  33. 2022Demographic Parity Constrained Minimax Optimal Regression under Linear Modelcited by 6
  34. 202410 Years of Fair Representations: Challenges and Opportunitiescited by 6
  35. 2021Unsupervised Information Obfuscation for Split Inference of Neural Networkscited by 5
  36. 2022Fair Group-Shared Representations with Normalizing Flowscited by 5
  37. 2022Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attackscited by 5
  38. 2021Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for Better Single-Source Domain Generalizationcited by 4

50 most cited of 111 All 111 (JSON)