mlsanityData to 7 Oct 2026

Set-Theoretic Compositionality of Sentence Embeddings

arXiv 2502.20975 ↗First posted 28 Feb 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Sentence encoders play a pivotal role in various NLP tasks; hence, an accurate evaluation of their compositional properties is paramount. However, existing evaluation methods predominantly focus on goal task-specific performance. This leaves a significant gap in understanding how well sentence embeddings demonstrate fundamental compositional properties in a task-independent context. Leveraging classical set theory, we address this gap by proposing six criteria based on three core "set-like" compositions/operations: textitTextOverlap, textitTextDifference, and textitTextUnion. We systematically evaluate 7 classical and 9 Large Language Model (LLM)-based sentence encoders to assess their alignment with these criteria. Our findings show that SBERT consistently demonstrates set-like compositional properties, surpassing even the latest LLMs. Additionally, we introduce a new dataset of ~192K samples designed to facilitate future benchmarking efforts on set-like compositionality of sentence embeddings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

39 of 57 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2014Distributed Representations of Sentences and Documentscited by 677
  12. 2024OLMo: Accelerating the Science of Language Modelscited by 599
27 more
  1. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  2. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  3. 2015Skip-Thought Vectorscited by 321
  4. 2017Toward Controlled Generation of Textcited by 271
  5. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  6. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  7. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  8. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  9. 2018An efficient framework for learning sentence representationscited by 197
  10. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  11. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  12. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 103
  13. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  14. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  15. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 90
  16. 2024OpenELM: An Efficient Language Model Family with Open Training and Inference Frameworkcited by 74
  17. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
  18. 2020Assessing Phrasal Representation and Composition in Transformerscited by 41
  19. 2019Evaluating Word Embedding Models: Methods and Experimental Resultscited by 37
  20. 2016Problems With Evaluation of Word Embeddings Using Word Similarity Taskscited by 25
  21. 2022Just Rank: Rethinking Evaluation with Word and Sentence Similaritiescited by 20
  22. 2021Sentence Bottleneck Autoencoders from Transformer Language Modelscited by 19
  23. 2024Revisiting Word Embeddings in the LLM Eracited by 13
  24. 2022Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Modelscited by 9
  25. 2023Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operationscited by 5
  26. 2024SetCSE: Set Operations using Contrastive Learning of Sentence Embeddingscited by 2
  27. 2019Hamming Sentence Embeddings for Information Retrievalcited by 1

Built on by

Not cited yet.