mlsanityData to 7 Oct 2026

What if This Modified That? Syntactic Interventions via Counterfactual Embeddings

arXiv 2105.14002 ↗First posted 28 May 2021cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

Neural language models exhibit impressive performance on a variety of tasks, but their internal reasoning may be difficult to understand. Prior art aims to uncover meaningful properties within model representations via probes, but it is unclear how faithfully such probes portray information that the models actually use. To overcome such limitations, we propose a technique, inspired by causal analysis, for generating counterfactual embeddings within models. In experiments testing our technique, we produce evidence that suggests some BERT-based models use a tree-distance-like representation of syntax in downstream prediction tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 28 May 2021.

Builds on

17 of 24 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  8. 2019Designing and Interpreting Probes with Control Taskscited by 608
  9. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  10. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  11. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  12. 2019Visualizing and Measuring the Geometry of BERTcited by 215
5 more
  1. 2020Information-Theoretic Probing for Linguistic Structurecited by 204
  2. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  3. 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
  4. 2020A Tale of a Probe and a Parsercited by 40
  5. 2019Linking artificial and human neural representations of languagecited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 16; 2024: 4; 2025: 7; 2026: 3.

39 papers.

Sort:

By area:

  1. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  2. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  4. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  5. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
  6. 2022Probing for the Usage of Grammatical Numbercited by 57
  7. 2023Break It Down: Evidence for Structural Compositionality in Neural Networkscited by 44
  8. 2023Beyond Surface Statistics: Scene Representations in a Latent Diffusion Modelcited by 29
  9. 2022Probing for Incremental Parse States in Autoregressive Language Modelscited by 24
  10. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  11. 2022Unit Testing for Concepts in Neural Networkscited by 23
  12. 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
27 more
  1. 2023Evaluating Neural Language Models as Cognitive Models of Language Acquisitioncited by 15
  2. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  3. 2023Operationalising Representation in Natural Language Processingcited by 14
  4. 2022When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probescited by 12
  5. 2023Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Numbercited by 12
  6. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  7. 2023Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)cited by 9
  8. 2024How Reliable are Causal Probing Interventions?cited by 9
  9. 2023More than Correlation: Do Large Language Models Learn Causal Representations of Space?cited by 7
  10. 2025Linguistic Interpretability of Transformer-based Language Models: a systematic reviewcited by 7
  11. 2025Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluationscited by 7
  12. 2025Analyzing the Inner Workings of Transformers in Compositional Generalizationcited by 6
  13. 2024A Rationale-centric Counterfactual Data Augmentation Method for Cross-Document Event Coreference Resolutioncited by 5
  14. 2022Prototype Based Classification from Hierarchy to Fairnesscited by 4
  15. 2023Competence-Based Analysis of Language Modelscited by 4
  16. 2025From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?cited by 2
  17. 2023Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddingscited by 1
  18. 2023On Modifying a Neural Network's Perceptioncited by 1
  19. 2023Semantic Composition in Visually Grounded Language Modelscited by 1
  20. 2025STARE at the Structure: Steering ICL Exemplar Selection with Structural Alignmentcited by 1
  21. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  22. 2026Perturbation: A simple and efficient adversarial tracer for representation learning in language modelscited by 1
  23. 2022Probe-Based Interventions for Modifying Agent Behaviorcited by 0
  24. 2022Automatically Generating Counterfactuals for Relation Classificationcited by 0
  25. 2025On the Performance of Concept Probing: The Influence of the Data (Extended Version)cited by 0
  26. 2025Concept Probing: Where to Find Human-Defined Concepts (Extended Version)cited by 0
  27. 2026Inference Time Causal Probing in LLMscited by 0