# Should Semantic Vector Composition be Explicit? Can it be Linear?

arXiv: https://arxiv.org/abs/2104.06555 · HTML: https://mlsanity.com/paper/2104.06555/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 82 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method](https://mlsanity.com/paper/1402.3722/) (2014)
- [Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion](https://mlsanity.com/paper/1804.07745/) (2018)
- [QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer](https://mlsanity.com/paper/2102.12846/) (2021)
- [UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus](https://mlsanity.com/paper/2010.10391/) (2020)
- [Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees](https://mlsanity.com/paper/2103.04350/) (2021)
- [Combining pre-trained language models and structured knowledge](https://mlsanity.com/paper/2101.12294/) (2021)
- [Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs](https://mlsanity.com/paper/1705.09189/) (2017)
- [The Frobenius anatomy of word meanings II: possessive relative pronouns](https://mlsanity.com/paper/1406.4690/) (2014)
- [Modelling Lexical Ambiguity with Density Matrices](https://mlsanity.com/paper/2010.05670/) (2020)
- [Sentence Entailment in Compositional Distributional Semantics](https://mlsanity.com/paper/1512.04419/) (2015)
- [Towards logical negation for compositional distributional semantics](https://mlsanity.com/paper/2005.04929/) (2020)
- [Evaluating Neural Word Representations in Tensor-Based Compositional Settings](https://mlsanity.com/paper/1408.6179/) (2014)
- [Attentive Tensor Product Learning](https://mlsanity.com/paper/1802.07089/) (2018)
- [At the Interface of Algebra and Statistics](https://mlsanity.com/paper/2004.05631/) (2020)
- [A Quantum Search Decoder for Natural Language Processing](https://mlsanity.com/paper/1909.05023/) (2019)
- [Quantum Mathematics in Artificial Intelligence](https://mlsanity.com/paper/2101.04255/) (2021)
- [Compositional Distributional Semantics with Compact Closed Categories and Frobenius Algebras](https://mlsanity.com/paper/1505.00138/) (2015)

## Built on by (most cited first, 3 of 3)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Semantic Wave Functions: Exploring Meaning in Large Language Models Through Quantum Formalism](https://mlsanity.com/paper/2503.10664/) (2025)
- [Developing a Foundation of Vector Symbolic Architectures Using Category Theory](https://mlsanity.com/paper/2501.05368/) (2025)

## Abstract (arXiv)

Vector representations have become a central element in semantic language modelling, leading to mathematical overlaps with many fields including quantum theory. Compositionality is a core goal for such representations: given representations for 'wet' and 'fish', how should the concept 'wet fish' be represented? This position paper surveys this question from two points of view. The first considers the question of whether an explicit mathematical representation can be successful using only tools from within linear algebra, or whether other mathematical tools are needed. The second considers whether semantic vector composition should be explicitly described mathematically, or whether it can be a model-internal side-effect of training a neural network. A third and newer question is whether a compositional model can be implemented on a quantum computer. Given the fundamentally linear nature of quantum mechanics, we propose that these questions are related, and that this survey may help to highlight candidate operations for future quantum implementation.