mlsanityData to 7 Oct 2026

Mediators in Determining what Processing BERT Performs First

arXiv 2104.06400 ↗First posted 13 Apr 2021cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Probing neural models for the ability to perform downstream tasks using their activation patterns is often used to localize what parts of the network specialize in performing what tasks. However, little work addressed potential mediating factors in such comparisons. As a test-case mediating factor, we consider the prediction's context length, namely the length of the span whose processing is minimally required to perform the prediction. We show that not controlling for context length may lead to contradictory conclusions as to the localization patterns of the network, depending on the distribution of the probing dataset. Indeed, when probing BERT with seven tasks, we find that it is possible to get 196 different rankings between them when manipulating the distribution of context lengths in the probing dataset. We conclude by presenting best practices for conducting such comparisons in the future.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 20 of its 28 references that are in the map. First posted on arXiv 13 Apr 2021.

Builds on

20 of 28 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  3. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  4. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  5. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  6. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  7. 2013Direct and Indirect Effectscited by 276
  8. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 215
  9. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  10. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  11. 2015Visualizing and Understanding Recurrent Networkscited by 158
  12. 2020Interpreting Graph Neural Networks for NLP With Differentiable Edge Maskingcited by 152
8 more
  1. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  2. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  3. 2019Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasetscited by 55
  4. 2016How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairscited by 39
  5. 2020jiant: A Software Toolkit for Research on General-Purpose Text Understanding Modelscited by 39
  6. 2018Deep RNNs Encode Soft Hierarchical Syntaxcited by 34
  7. 2019On the Linguistic Representational Power of Neural Machine Translation Modelscited by 34
  8. 2019Automatically Extracting Challenge Sets for Non local Phenomena in Neural Machine Translationcited by 15

Built on by

Cited by year

Citing papers per year: 2021: 7; 2022: 3; 2023: 4.

14 papers.

Sort:

  1. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  2. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 58
  3. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  4. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54
  5. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  6. 2021The Grammar-Learning Trajectories of Neural Language Modelscited by 36
  7. 2023Weight subcloning: direct initialization of transformers using larger pretrained onescited by 30
  8. 2022Predicting Fine-Tuning Performance with Probingcited by 13
  9. 2022Controlled Text Reductioncited by 10
  10. 2021Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowordscited by 9
  11. 2021Semantics-aware Attention Improves Neural Machine Translationcited by 5
  12. 2021ComSum: Commit Messages Summarization and Meaning Preservationcited by 3
2 more
  1. 2021On Neurons Invariant to Sentence Structural Changes in Neural Machine Translationcited by 2
  2. 2023MuLER: Detailed and Scalable Reference-based Evaluationcited by 0