mlsanityData to 14 Oct 2026

Preserving Task-Relevant Information Under Linear Concept Removal

arXiv 2506.10703 ↗First posted 12 Jun 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Modern neural networks often encode unwanted concepts alongside task-relevant information, leading to fairness and interpretability concerns. Existing post-hoc approaches can remove undesired concepts but often degrade useful signals. We introduce SPLINCE-Simultaneous Projection for LINear concept removal and Covariance prEservation - which eliminates sensitive concepts from representations while exactly preserving their covariance with a target label. SPLINCE achieves this via an oblique projection that 'splices out' the unwanted direction yet protects important label correlations. Theoretically, it is the unique solution that removes linear concept predictability and maintains target covariance with minimal embedding distortion. Empirically, SPLINCE outperforms baselines on benchmarks such as Bias in Bios and Winobias, removing protected attributes while minimally damaging main-task information.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

34 of 45 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  6. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  10. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  11. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  12. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
22 more
  1. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 381
  2. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  3. 2023Improving Text Embeddings with Large Language Modelscited by 303
  4. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  5. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
  6. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  7. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  8. 2022Linear Adversarial Concept Erasurecited by 101
  9. 2016Places: An Image Database for Deep Scene Understandingcited by 76
  10. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  11. 2020OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddingscited by 40
  12. 2021Contrastive Learning for Fair Representationscited by 34
  13. 2022Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Informationcited by 34
  14. 2023A Geometric Notion of Causal Probingcited by 25
  15. 2023Debiasing Algorithm through Model Adaptationcited by 24
  16. 2023Efficient fair PCA for fair representation learningcited by 23
  17. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmarkcited by 19
  18. 2022Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projectioncited by 17
  19. 2024Gumbel Counterfactual Generation From Language Modelscited by 14
  20. 2023Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimationcited by 6
  21. 2024Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppressioncited by 5
  22. 2025Dual Debiasing: Remove Stereotypes and Keep Factual Gender for Fair Language Modeling and Translationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Modelscited by 1
  2. 2026Interpolation Is Not Invariance: Pair Count Is Not Coverage in Transformation Auditscited by 0
  3. 2026Measurement-Gated Provenance Attenuation for Frozen EEG Representationscited by 0
  4. 2026Operational Abstractions of Neural Network Concepts via Topological Representationscited by 0