# Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models

arXiv: https://arxiv.org/abs/2304.13803 · HTML: https://mlsanity.com/paper/2304.13803/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 15 of 15)

- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models](https://mlsanity.com/paper/2204.08110/) (2022)
- [Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM](https://mlsanity.com/paper/2303.01911/) (2023)
- [XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization](https://mlsanity.com/paper/2010.06478/) (2020)
- [It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT](https://mlsanity.com/paper/2010.08275/) (2020)
- [Improving Word Translation via Two-Stage Contrastive Learning](https://mlsanity.com/paper/2203.08307/) (2022)
- [Handling Homographs in Neural Machine Translation](https://mlsanity.com/paper/1708.06510/) (2017)
- [Multilingual Word Sense Disambiguation with Unified Sense Representation](https://mlsanity.com/paper/2210.07447/) (2022)
- [Semi-Supervised and Unsupervised Sense Annotation via Translations](https://mlsanity.com/paper/2106.06462/) (2021)

## Built on by (most cited first, 2 of 2)

- [Can LLMs assist with Ambiguity? A Quantitative Evaluation of various Large Language Models on Word Sense Disambiguation](https://mlsanity.com/paper/2411.18337/) (2024)
- [An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs](https://mlsanity.com/paper/2603.05400/) (2026)

## Abstract (arXiv)

Pretrained Language Models (PLMs) learn rich cross-lingual knowledge and can be finetuned to perform well on diverse tasks such as translation and multilingual word sense disambiguation (WSD). However, they often struggle at disambiguating word sense in a zero-shot setting. To better understand this contrast, we present a new study investigating how well PLMs capture cross-lingual word sense with Contextual Word-Level Translation (C-WLT), an extension of word-level translation that prompts the model to translate a given word in context. We find that as the model size increases, PLMs encode more cross-lingual word sense knowledge and better use context to improve WLT performance. Building on C-WLT, we introduce a zero-shot approach for WSD, tested on 18 languages from the XL-WSD dataset. Our method outperforms fully supervised baselines on recall for many evaluation languages without additional training or finetuning. This study presents a first step towards understanding how to best leverage the cross-lingual knowledge inside PLMs for robust zero-shot reasoning in any language.