# Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions

arXiv: https://arxiv.org/abs/2305.07303 · HTML: https://mlsanity.com/paper/2305.07303/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 12 May 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Poincar'e GloVe: Hyperbolic Word Embeddings](https://mlsanity.com/paper/1810.06546/) (2018)
- [Multi-relational Poincar'e Graph Embeddings](https://mlsanity.com/paper/1905.09791/) (2019)
- [Definition Modeling: Learning to define word embeddings in natural language](https://mlsanity.com/paper/1612.00394/) (2016)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://mlsanity.com/paper/1608.00869/) (2016)
- [Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation](https://mlsanity.com/paper/1906.10007/) (2019)
- [Conditional Generators of Words Definitions](https://mlsanity.com/paper/1806.10090/) (2018)
- [Learning to Understand Phrases by Embedding the Dictionary](https://mlsanity.com/paper/1504.00548/) (2015)
- [Problems With Evaluation of Word Embeddings Using Word Similarity Tasks](https://mlsanity.com/paper/1605.02276/) (2016)
- [Skip-gram word embeddings in hyperbolic space](https://mlsanity.com/paper/1809.01498/) (2018)
- [DefSent: Sentence Embeddings using Definition Sentences](https://mlsanity.com/paper/2105.04339/) (2021)
- [Semeval-2022 Task 1: CODWOE -- Comparing Dictionaries and Word Embeddings](https://mlsanity.com/paper/2205.13858/) (2022)
- [Learning Disentangled Representations for Natural Language Definitions](https://mlsanity.com/paper/2210.02898/) (2022)
- [Categorization of Semantic Roles for Dictionary Definitions](https://mlsanity.com/paper/1806.07711/) (2018)
- [Building a Knowledge Graph from Natural Language Definitions for Interpretable Text Entailment Recognition](https://mlsanity.com/paper/1806.07731/) (2018)

## Built on by (most cited first, 4 of 4)

- [HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts](https://mlsanity.com/paper/2404.17507/) (2024)
- [Multi-Operational Mathematical Derivations in Latent Space](https://mlsanity.com/paper/2311.01230/) (2023)
- [Model Editing with Graph-Based External Memory](https://mlsanity.com/paper/2505.18343/) (2025)
- [Principles of Concept Representation in Sentence Encoders](https://mlsanity.com/paper/2606.06994/) (2026)

## Abstract (arXiv)

Natural language definitions possess a recursive, self-explanatory semantic structure that can support representation learning methods able to preserve explicit conceptual relations and constraints in the latent space. This paper presents a multi-relational model that explicitly leverages such a structure to derive word embeddings from definitions. By automatically extracting the relations linking defined and defining terms from dictionaries, we demonstrate how the problem of learning word embeddings can be formalised via a translational framework in Hyperbolic space and used as a proxy to capture the global semantic structure of definitions. An extensive empirical analysis demonstrates that the framework can help imposing the desired structural constraints while preserving the semantic mapping required for controllable and interpretable traversal. Moreover, the experiments reveal the superiority of the Hyperbolic word embeddings over the Euclidean counterparts and demonstrate that the multi-relational approach can obtain competitive results when compared to state-of-the-art neural models, with the advantage of being intrinsically more efficient and interpretable.