mlsanityData to 14 Oct 2026

Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment

arXiv 2609.01322 ↗First posted 1 Sep 2026cs.CL cs.LGcited by 0

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff in constructing text representations for adjustment: they must be sufficiently large and/or dense to preserve the confounding variables necessary for unbiased effect estimation, but sufficiently small and/or sparse to satisfy finite-sample overlap and yield low-variance estimates. To address this tradeoff, we turn to sparse autoencoders (SAEs), and propose a novel causal adjustment pipeline that iteratively selects a minimal set of SAE features via conditional independence tests. We find that SAE representations achieve better adjustments (lower bias and and higher coverage) than alternative representations in standard semi-synthetic evaluations with binary confounders, and their interpretability offers opportunities for falsification. We also introduce a more realistic semi-synthetic evaluation that uses multi-label data as the unobserved confounders and find off-the-shelf adjustment methods require increased investigation for these more complex settings. Code: https://github.com/mianzg/sae-text-confounder

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 12 of its 30 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

12 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 11

  1. 2013k-Sparse Autoencoderscited by 215
  2. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  3. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
  4. 2017Double/Debiased/Neyman Machine Learning of Treatment Effectscited by 76
  5. 2020Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimatescited by 59
  6. 2018Challenges of Using Text Classifiers for Causal Inferencecited by 33
  7. 2019Characterization of Overlap in Observational Studiescited by 15
  8. 2023RCT Rejection Sampling for Causal Estimation Evaluationcited by 7
  9. 2024Proximal Causal Inference With Text Datacited by 7
  10. 2025ConceptScope: Characterizing Dataset Bias via Disentangled Visual Conceptscited by 5
  11. 2025Adjustment for Confounding using Pre-Trained Representationscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Causal Effect Estimation with Latent Textual Treatments
  2. 2025Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models
  3. 2025Exploratory Causal Inference in SAEnce
  4. 2026Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models
  5. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
  6. 2025A Technical Exploration of Causal Inference with Hybrid LLM Synthetic Data
  7. 2025Technical Report: Facilitating the Adoption of Causal Inference Methods Through LLM-Empowered Co-Pilot
  8. 2026ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding