# Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information

arXiv: https://arxiv.org/abs/2305.01788 · HTML: https://mlsanity.com/paper/2305.01788/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [FLAVA: A Foundational Language And Vision Alignment Model](https://mlsanity.com/paper/2112.04482/) (2021)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://mlsanity.com/paper/1908.07245/) (2019)
- [Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrieval](https://mlsanity.com/paper/2204.09868/) (2022)
- [Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders](https://mlsanity.com/paper/2005.02590/) (2020)
- [Image-text Retrieval: A Survey on Recent Research and Development](https://mlsanity.com/paper/2203.14713/) (2022)
- [DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models](https://mlsanity.com/paper/2210.10606/) (2022)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses](https://mlsanity.com/paper/2104.14694/) (2021)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback](https://mlsanity.com/paper/2106.04128/) (2021)
- [MedJEx: A Medical Jargon Extraction Model with Wiki's Hyperlink Span and Contextualized Masked Language Model Score](https://mlsanity.com/paper/2210.05875/) (2022)
- [DALLE-URBAN: Capturing the urban design expertise of large text to image transformers](https://mlsanity.com/paper/2208.04139/) (2022)
- [Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings](https://mlsanity.com/paper/1603.09188/) (2016)
- [Cross-lingual Visual Verb Sense Disambiguation](https://mlsanity.com/paper/1904.05092/) (2019)
- [Learning from Lexical Perturbations for Consistent Visual Question Answering](https://mlsanity.com/paper/2011.13406/) (2020)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)
- [Knowledge-based Word Sense Disambiguation using Topic Models](https://mlsanity.com/paper/1801.01900/) (2018)
- [Transductive Visual Verb Sense Disambiguation](https://mlsanity.com/paper/2012.10821/) (2020)

## Built on by (most cited first, 5 of 5)

- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://mlsanity.com/paper/2306.12424/) (2023)
- [Is it safe to cross? Interpretable Risk Assessment with GPT-4V for Safety-Aware Street Crossing](https://mlsanity.com/paper/2402.06794/) (2024)
- [ODD: A Benchmark Dataset for the Natural Language Processing based Opioid Related Aberrant Behavior Detection](https://mlsanity.com/paper/2307.02591/) (2023)
- [Quantum Visual Word Sense Disambiguation: Unraveling Ambiguities Through Quantum Inference Model](https://mlsanity.com/paper/2512.24687/) (2025)
- [Bridging Lexical Ambiguity and Vision: A Mini Review on Visual Word Sense Disambiguation](https://mlsanity.com/paper/2602.01193/) (2026)

## Abstract (arXiv)

Visual Word Sense Disambiguation (VWSD) is a task to find the image that most accurately depicts the correct sense of the target word for the given context. Previously, image-text matching models often suffered from recognizing polysemous words. This paper introduces an unsupervised VWSD approach that uses gloss information of an external lexical knowledge-base, especially the sense definitions. Specifically, we suggest employing Bayesian inference to incorporate the sense definitions when sense information of the answer is not provided. In addition, to ameliorate the out-of-dictionary (OOD) issue, we propose a context-aware definition generation with GPT-3. Experimental results show that the VWSD performance significantly increased with our Bayesian inference-based approach. In addition, our context-aware definition generation achieved prominent performance improvement in OOD examples exhibiting better performance than the existing definition generation method.