# Interventional Probing in High Dimensions: An NLI Case Study

arXiv: https://arxiv.org/abs/2304.10346 · HTML: https://mlsanity.com/paper/2304.10346/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://mlsanity.com/paper/2005.00719/) (2020)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [CausaLM: Causal Model Explanation Through Counterfactual Language Models](https://mlsanity.com/paper/2005.13407/) (2020)
- [Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information](https://mlsanity.com/paper/1808.08079/) (2018)
- [Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations](https://mlsanity.com/paper/2106.12723/) (2021)
- [HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning](https://mlsanity.com/paper/1904.12166/) (2019)
- [An information theoretic view on selecting linguistic probes](https://mlsanity.com/paper/2009.07364/) (2020)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [Supporting Context Monotonicity Abstractions in Neural NLI Models](https://mlsanity.com/paper/2105.08008/) (2021)
- [Decomposing Natural Logic Inferences in Neural NLI](https://mlsanity.com/paper/2112.08289/) (2021)

## Built on by (most cited first, 5 of 5)

- [Probing the Category of Verbal Aspect in Transformer Language Models](https://mlsanity.com/paper/2406.02335/) (2024)
- [Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluation](https://mlsanity.com/paper/2305.17819/) (2023)
- [Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension](https://mlsanity.com/paper/2608.10566/) (2026)
- [Enhancing Robustness in Biomedical NLI Models: A Probing Approach for Clinical Trials](https://mlsanity.com/paper/2402.02558/) (2024)
- [Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models](https://mlsanity.com/paper/2404.02622/) (2024)

## Abstract (arXiv)

Probing strategies have been shown to detect the presence of various linguistic features in large language models; in particular, semantic features intermediate to the "natural logic" fragment of the Natural Language Inference task (NLI). In the case of natural logic, the relation between the intermediate features and the entailment label is explicitly known: as such, this provides a ripe setting for interventional studies on the NLI models' representations, allowing for stronger causal conjectures and a deeper critical analysis of interventional probing methods. In this work, we carry out new and existing representation-level interventions to investigate the effect of these semantic features on NLI classification: we perform amnesic probing (which removes features as directed by learned linear probes) and introduce the mnestic probing variation (which forgets all dimensions except the probe-selected ones). Furthermore, we delve into the limitations of these methods and outline some pitfalls have been obscuring the effectivity of interventional probing studies.