# Topics to Avoid: Demoting Latent Confounds in Text Classification

arXiv: https://arxiv.org/abs/1909.00453 · HTML: https://mlsanity.com/paper/1909.00453/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2019.

## Built on by (most cited first, 14 of 14)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers](https://mlsanity.com/paper/2103.12279/) (2021)
- [Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates](https://mlsanity.com/paper/2110.03212/) (2021)
- [Dataset Geography: Mapping Language Data to Language Users](https://mlsanity.com/paper/2112.03497/) (2021)
- [How to keep text private? A systematic review of deep learning methods for privacy-preserving natural language processing](https://mlsanity.com/paper/2205.10095/) (2022)
- [Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning](https://mlsanity.com/paper/2105.14241/) (2021)
- [Mitigating Gender Bias in Machine Translation through Adversarial Learning](https://mlsanity.com/paper/2203.10675/) (2022)
- [Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching](https://mlsanity.com/paper/2203.08979/) (2022)
- [The Knesset Corpus: An Annotated Corpus of Hebrew Parliamentary Proceedings](https://mlsanity.com/paper/2405.18115/) (2024)
- [Quantifying the Task-Specific Information in Text-Based Classifications](https://mlsanity.com/paper/2110.08931/) (2021)
- [PASTRIE: A Corpus of Prepositions Annotated with Supersense Tags in Reddit International English](https://mlsanity.com/paper/2110.12243/) (2021)
- [CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models](https://mlsanity.com/paper/2509.05230/) (2025)

## Abstract (arXiv)

Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. In this work, we observe this limitation with respect to the task of native language identification. We find that standard text classifiers which perform well on the test set end up learning topical features which are confounds of the prediction task (e.g., if the input text mentions Sweden, the classifier predicts that the author's native language is Swedish). We propose a method that represents the latent topical confounds and a model which "unlearns" confounding features by predicting both the label of the input text and the confound; but we train the two predictors adversarially in an alternating fashion to learn a text representation that predicts the correct label but is less prone to using information about the confound. We show that this model generalizes better and learns features that are indicative of the writing style rather than the content.