# Improving Interpretability of Word Embeddings by Generating Definition and Usage

arXiv: https://arxiv.org/abs/1912.05898 · HTML: https://mlsanity.com/paper/1912.05898/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 5 of 5)

- [Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings](https://mlsanity.com/paper/2205.13858/) (2022)
- [DORE: A Dataset For Portuguese Definition Generation](https://mlsanity.com/paper/2403.18018/) (2024)
- [IRB-NLP at SemEval-2022 Task 1: Exploring the Relationship Between Words and Their Semantic Representations](https://mlsanity.com/paper/2205.06840/) (2022)
- [Vec2Gloss: definition modeling leveraging contextualized vectors with Wordnet gloss](https://mlsanity.com/paper/2305.17855/) (2023)
- [A Scalable Space-efficient In-database Interpretability Framework for Embedding-based Semantic SQL Queries](https://mlsanity.com/paper/2302.12178/) (2023)

## Abstract (arXiv)

Word embeddings are substantially successful in capturing semantic relations among words. However, these lexical semantics are difficult to be interpreted. Definition modeling provides a more intuitive way to evaluate embeddings by utilizing them to generate natural language definitions of corresponding words. This task is of great significance for practical application and in-depth understanding of word representations. We propose a novel framework for definition modeling, which can generate reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and study whether it is possible to utilize embeddings to generate example sentences of words. These ways are a more direct and explicit expression of embedding's semantics for better interpretability. We extend the single task model to multi-task setting and investigate several joint multi-task models to combine usage modeling and definition modeling together. Experimental results on existing Oxford dataset and a new collected Oxford-2019 dataset show that our single-task model achieves the state-of-the-art result in definition modeling and the multi-task learning methods are helpful for two tasks to improve the performance.