# IRB-NLP at SemEval-2022 Task 1: Exploring the Relationship Between Words and Their Semantic Representations

arXiv: https://arxiv.org/abs/2205.06840 · HTML: https://mlsanity.com/paper/2205.06840/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [Learning to Understand Phrases by Embedding the Dictionary](https://mlsanity.com/paper/1504.00548/) (2015)
- [Mark my Word: A Sequence-to-Sequence Approach to Definition Modeling](https://mlsanity.com/paper/1911.05715/) (2019)
- [Incorporating Sememes into Chinese Definition Modeling](https://mlsanity.com/paper/1905.06512/) (2019)
- [BERT for Monolingual and Cross-Lingual Reverse Dictionary](https://mlsanity.com/paper/2009.14790/) (2020)
- [Implementing a Reverse Dictionary, based on word definitions, using a Node-Graph Architecture](https://mlsanity.com/paper/1606.00025/) (2016)
- [Improving Interpretability of Word Embeddings by Generating Definition and Usage](https://mlsanity.com/paper/1912.05898/) (2019)
- [Evaluating a Multi-sense Definition Generation Model for Multiple Languages](https://mlsanity.com/paper/2006.07398/) (2020)
- [Multi-sense Definition Modeling using Word Sense Decompositions](https://mlsanity.com/paper/1909.09483/) (2019)
- [PREDICT: Persian Reverse Dictionary](https://mlsanity.com/paper/2105.00309/) (2021)

## Built on by (most cited first, 1 of 1)

- [Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings](https://mlsanity.com/paper/2205.13858/) (2022)

## Abstract (arXiv)

What is the relation between a word and its description, or a word and its embedding? Both descriptions and embeddings are semantic representations of words. But, what information from the original word remains in these representations? Or more importantly, which information about a word do these two representations share? Definition Modeling and Reverse Dictionary are two opposite learning tasks that address these questions. The goal of the Definition Modeling task is to investigate the power of information laying inside a word embedding to express the meaning of the word in a humanly understandable way -- as a dictionary definition. Conversely, the Reverse Dictionary task explores the ability to predict word embeddings directly from its definition. In this paper, by tackling these two tasks, we are exploring the relationship between words and their semantic representations. We present our findings based on the descriptive, exploratory, and predictive data analysis conducted on the CODWOE dataset. We give a detailed overview of the systems that we designed for Definition Modeling and Reverse Dictionary tasks, and that achieved top scores on SemEval-2022 CODWOE challenge in several subtasks. We hope that our experimental results concerning the predictive models and the data analyses we provide will prove useful in future explorations of word representations and their relationships.