mlsanityData to 7 Oct 2026

What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories

arXiv 2302.03353 ↗First posted 7 Feb 2023cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Language Models are the core for almost any Natural Language Processing system nowadays. One of their particularities is their contextualized representations, a game changer feature when a disambiguation between word senses is necessary. In this paper we aim to explore to what extent language models are capable of discerning among senses at inference time. We performed this analysis by prompting commonly used Languages Models such as BERT or RoBERTa to perform the task of Word Sense Disambiguation (WSD). We leverage the relation between word senses and domains, and cast WSD as a textual entailment problem, where the different hypothesis refer to the domains of the word senses. Our results show that this approach is indeed effective, close to supervised systems.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 34 of its 76 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

34 of 76 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  10. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  11. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  12. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
22 more
  1. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  2. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  3. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  4. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  5. 2021How Many Data Points is a Prompt Worth?cited by 197
  6. 2021Entailment as Few-Shot Learnercited by 122
  7. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  8. 2019GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledgecited by 78
  9. 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
  10. 2020Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoderscited by 61
  11. 2020Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Startcited by 44
  12. 2019Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguationcited by 40
  13. 2021Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and Systemcited by 30
  14. 2022Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learningcited by 25
  15. 2019Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representationscited by 21
  16. 2022Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inferencecited by 17
  17. 2020Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentencescited by 14
  18. 2021Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Modelscited by 13
  19. 2022ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizationscited by 7
  20. 2022Probing via Promptingcited by 7
  21. 2022Zero-Shot Aspect-Based Sentiment Analysiscited by 5
  22. 2022SEE-Few: Seed, Expand and Entail for Few-shot Named Entity Recognitioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 2; 2026: 2.

9 papers.

  1. 2024The broader spectrum of in-context learningcited by 33
  2. 2023Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Datasetcited by 17
  3. 2024Can LLMs assist with Ambiguity? A Quantitative Evaluation of various Large Language Models on Word Sense Disambiguationcited by 2
  4. 2026In the LLM era, Word Sense Induction remains unsolvedcited by 2
  5. 2024LLMRS: Unlocking Potentials of LLM-Based Recommender Systems for Software Purchasecited by 1
  6. 2024Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognitioncited by 1
  7. 2025Prompt Balance Matters: Understanding How Imbalanced Few-Shot Learning Affects Multilingual Sense Disambiguation in LLMscited by 1
  8. 2025DACE For Railway Acronym Disambiguationcited by 0
  9. 2026SwanNLP at SemEval-2026 Task 5: An LLM-based Framework for Plausibility Scoring in Narrative Word Sense Disambiguationcited by 0