mlsanityData to 7 Oct 2026

What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?

arXiv 2007.06897 ↗First posted 14 Jul 2020cs.CL cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

We evaluate named entity representations of BERT-based NLP models by investigating their robustness to replacements from the same typed class in the input. We highlight that on several tasks while such perturbations are natural, state of the art trained models are surprisingly brittle. The brittleness continues even with the recent entity-aware BERT models. We also try to discern the cause of this non-robustness, considering factors such as tokenization and frequency of occurrence. Then we provide a simple method that ensembles predictions from multiple replacements while jointly modeling the uncertainty of type annotations and label predictions. Experiments on three NLP tasks show that our method enhances robustness and increases accuracy on both natural and adversarial datasets.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 6; 2023: 1; 2024: 2; 2025: 2.

17 papers.

Sort:

By area:

  1. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 48
  2. 2022Prompting Language Models for Linguistic Structurecited by 39
  3. 2022Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lenscited by 28
  4. 2021Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognitioncited by 19
  5. 2021On the Robustness of Reading Comprehension Models to Entity Renamingcited by 17
  6. 2021Inspecting the concept knowledge graph encoded by modern language modelscited by 15
  7. 2022Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answeringcited by 6
  8. 2021Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attackcited by 3
  9. 2022Can Language Representation Models Think in Bets?cited by 3
  10. 2021NATURE: Natural Auxiliary Text Utterances for Realistic Spoken Language Evaluationcited by 2
  11. 2022Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probescited by 2
  12. 2023Construction Grammar and Language Modelscited by 2
5 more
  1. 2024EvoGrad: A Dynamic Take on the Winograd Schema Challenge with Human Adversariescited by 2
  2. 2024Topic Aware Probing: From Sentence Length Prediction to Idiom Identification how reliant are Neural Language Models on Topic?cited by 2
  3. 2021Novel Aficionados and Doppelg"angers: a referential task for semantic representations of individual entitiescited by 0
  4. 2025Do We Really Need GNNs with Explicit Structural Modeling? MLPs Suffice for Language Model Representationscited by 0
  5. 2025From Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generationcited by 0