# Subspace Representations for Soft Set Operations and Sentence Similarities

arXiv: https://arxiv.org/abs/2210.13034 · HTML: https://mlsanity.com/paper/2210.13034/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [Word Rotator's Distance](https://mlsanity.com/paper/2004.15003/) (2020)
- [Evaluating Gender Bias in Natural Language Inference](https://mlsanity.com/paper/2105.05541/) (2021)
- [Learning Aggregation Functions](https://mlsanity.com/paper/2012.08482/) (2020)
- [Multimodal Word Distributions](https://mlsanity.com/paper/1704.08424/) (2017)
- [Don't Settle for Average, Go for the Max: Fuzzy Sets and Max-Pooled Word Vectors](https://mlsanity.com/paper/1904.13264/) (2019)

## Built on by (most cited first, 2 of 2)

- [What Machine Learning Tells Us About the Mathematical Structure of Concepts](https://mlsanity.com/paper/2408.15507/) (2024)
- [Splits! Flexible Sociocultural Linguistic Investigation at Scale](https://mlsanity.com/paper/2504.04640/) (2025)

## Abstract (arXiv)

In the field of natural language processing (NLP), continuous vector representations are crucial for capturing the semantic meanings of individual words. Yet, when it comes to the representations of sets of words, the conventional vector-based approaches often struggle with expressiveness and lack the essential set operations such as union, intersection, and complement. Inspired by quantum logic, we realize the representation of word sets and corresponding set operations within pre-trained word embedding spaces. By grounding our approach in the linear subspaces, we enable efficient computation of various set operations and facilitate the soft computation of membership functions within continuous spaces. Moreover, we allow for the computation of the F-score directly within word vectors, thereby establishing a direct link to the assessment of sentence similarity. In experiments with widely-used pre-trained embeddings and benchmarks, we show that our subspace-based set operations consistently outperform vector-based ones in both sentence similarity and set retrieval tasks.