# Modelling Monotonic and Non-Monotonic Attribute Dependencies with Embeddings: A Theoretical Analysis

arXiv: https://arxiv.org/abs/2106.14431 · HTML: https://mlsanity.com/paper/2106.14431/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Hyperbolic Entailment Cones for Learning Hierarchical Embeddings](https://mlsanity.com/paper/1804.01882/) (2018)
- [Order-Embeddings of Images and Language](https://mlsanity.com/paper/1511.06361/) (2015)
- [Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach](https://mlsanity.com/paper/1706.05674/) (2017)
- [Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures](https://mlsanity.com/paper/1805.06627/) (2018)
- [Spherical Text Embedding](https://mlsanity.com/paper/1911.01196/) (2019)
- [From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules](https://mlsanity.com/paper/1805.10461/) (2018)
- [A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors](https://mlsanity.com/paper/1805.05388/) (2018)
- [Nonparametric Spherical Topic Modeling with Word Embeddings](https://mlsanity.com/paper/1604.00126/) (2016)
- [Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)](https://mlsanity.com/paper/1606.02979/) (2016)
- [Efficient Correlated Topic Modeling with Topic Embedding](https://mlsanity.com/paper/1707.00206/) (2017)

## Built on by (most cited first, 1 of 1)

- [Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge](https://mlsanity.com/paper/2210.05723/) (2022)

## Abstract (arXiv)

During the last decade, entity embeddings have become ubiquitous in Artificial Intelligence. Such embeddings essentially serve as compact but semantically meaningful representations of the entities of interest. In most approaches, vectors are used for representing the entities themselves, as well as for representing their associated attributes. An important advantage of using attribute embeddings is that (some of the) semantic dependencies between the attributes can thus be captured. However, little is known about what kinds of semantic dependencies can be modelled in this way. The aim of this paper is to shed light on this question, focusing on settings where the embedding of an entity is obtained by pooling the embeddings of its known attributes. Our particular focus is on studying the theoretical limitations of different embedding strategies, rather than their ability to effectively learn attribute dependencies in practice. We first show a number of negative results, revealing that some of the most popular embedding models are not able to capture even basic Horn rules. However, we also find that some embedding strategies are capable, in principle, of modelling both monotonic and non-monotonic attribute dependencies.