mlsanityData to 7 Oct 2026

Controlling Directions Orthogonal to a Classifier

arXiv 2201.11259 ↗First posted 27 Jan 2022cs.LG cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We propose to identify directions invariant to a given classifier so that these directions can be controlled in tasks such as style transfer. While orthogonal decomposition is directly identifiable when the given classifier is linear, we formally define a notion of orthogonality in the non-linear case. We also provide a surprisingly simple method for constructing the orthogonal classifier (a classifier utilizing directions other than those of the given classifier). Empirically, we present three use cases where controlling orthogonal variation is important: style transfer, domain adaptation, and fairness. The orthogonal classifier enables desired style transfer when domains vary in multiple aspects, improves domain adaptation with label shifts and mitigates the unfairness as a predictor. The code is available at http://github.com/Newbeeer/orthogonal_classifier

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 27 Jan 2022.

Builds on

30 of 44 references.

References · 30

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  2. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  3. 2019Invariant Risk Minimizationcited by 1,832
  4. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  5. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  6. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  7. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  8. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  9. 2018Disentangling by Factorisingcited by 614
  10. 2018Learning Adversarially Fair and Transferable Representationscited by 353
  11. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  12. 2018Towards a Definition of Disentangled Representationscited by 294
18 more
  1. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  2. 2021Gradient Matching for Domain Generalizationcited by 258
  3. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  4. 2018A DIRT-T Approach to Unsupervised Domain Adaptationcited by 215
  5. 2019Learning De-biased Representations with Biased Representationscited by 199
  6. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  7. 2020Domain Adaptation with Conditional Distribution Matching and Generalized Label Shiftcited by 124
  8. 2019Regularized Learning for Domain Adaptation under Label Shiftscited by 118
  9. 2018Learning Controllable Fair Representationscited by 83
  10. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  11. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  12. 2020Shape-Texture Debiased Neural Network Trainingcited by 76
  13. 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
  14. 2019Ensembles of Locally Independent Prediction Modelscited by 22
  15. 2021Learning Representations that Support Robust Transfer of Predictorscited by 17
  16. 2014Dropout Rademacher Complexity of Deep Neural Networkscited by 14
  17. 2018Learning Qualitatively Diverse and Interpretable Rules for Classificationcited by 10
  18. 2015The Multiverse Loss for Robust Transfer Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 6; 2024: 3; 2025: 1.

13 papers.

Sort:

By area:

  1. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  2. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  3. 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotationscited by 30
  4. 2022Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refittingcited by 26
  5. 2023Simplicity Bias in 1-Hidden Layer Neural Networkscited by 26
  6. 2023Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Featurescited by 9
  7. 2023Scalable Infomin Learningcited by 8
  8. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  9. 2023Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Toolscited by 7
  10. 2023Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shiftscited by 5
  11. 2022Learning an Invertible Output Mapping Can Mitigate Simplicity Bias in Neural Networkscited by 3
  12. 2024Generalizing Orthogonalization for Models with Non-Linearitiescited by 0
1 more
  1. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0