mlsanityData to 7 Oct 2026

WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations

arXiv 1808.09121 ↗First posted 28 Aug 2018cs.CLcited by 370

Not on the map. Why?

Abstract (arXiv)

By design, word embeddings are unable to model the dynamic nature of words' semantics, i.e., the property of words to correspond to potentially different meanings. To address this limitation, dozens of specialized meaning representation techniques such as sense or contextualized embeddings have been proposed. However, despite the popularity of research on this topic, very few evaluation benchmarks exist that specifically focus on the dynamic semantics of words. In this paper we show that existing models have surpassed the performance ceiling of the standard evaluation dataset for the purpose, i.e., Stanford Contextual Word Similarity, and highlight its shortcomings. To address the lack of a suitable benchmark, we put forward a large-scale Word in Context dataset, called WiC, based on annotations curated by experts, for generic evaluation of context-sensitive representations. WiC is released in https://pilehvar.github.io/wic/.

Key facts. Cited by 370 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 60; 2023: 77; 2024: 83; 2025: 66; 2026: 39.

370 papers.

Sort:

By area:

  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2022Emergent Abilities of Large Language Modelscited by 2,483
  3. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  6. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  7. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  8. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
  9. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  10. 2021MetaICL: Learning to Learn In Contextcited by 450
  11. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  12. 2023Symbolic Discovery of Optimization Algorithmscited by 423
38 more
  1. 2023Adapting Language Models to Compress Contextscited by 350
  2. 2021True Few-Shot Learning with Language Modelscited by 343
  3. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
  4. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  5. 2023Can Large Language Models Transform Computational Social Science?cited by 245
  6. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  7. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  8. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  9. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  10. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  11. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  12. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  13. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  14. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  15. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  16. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  17. 2023Full Parameter Fine-tuning for Large Language Models with Limited Resourcescited by 134
  18. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  19. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  20. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  21. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  22. 2022Fusing finetuned models for better pretrainingcited by 110
  23. 2024EMR-Merging: Tuning-Free High-Performance Model Mergingcited by 108
  24. 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 105
  25. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  26. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  27. 2021Improving and Simplifying Pattern Exploiting Trainingcited by 97
  28. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  29. 2022Language Models are General-Purpose Interfacescited by 96
  30. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 91
  31. 2023DoGE: Domain Reweighting with Generalization Estimationcited by 87
  32. 2022ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Promptscited by 85
  33. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  34. 2022Revisiting Parameter-Efficient Tuning: Are We Really There Yet?cited by 76
  35. 2024Parameter Competition Balancing for Model Mergingcited by 71
  36. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 67
  37. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 66
  38. 2023Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterizationcited by 64

50 most cited of 370 All 370 (JSON)