Space Decomposition for Sentence Embedding
Not on the map. Why?
Abstract (arXiv)
Determining sentence pair similarity is crucial for various NLP tasks. A common technique to address this is typically evaluated on a continuous semantic textual similarity scale from 0 to 5. However, based on a linguistic observation in STS annotation guidelines, we found that the score in the range [4,5] indicates an upper-range sample, while the rest are lower-range samples. This necessitates a new approach to treating the upper-range and lower-range classes separately. In this paper, we introduce a novel embedding space decomposition method called MixSP utilizing a Mixture of Specialized Projectors, designed to distinguish and rank upper-range and lower-range samples accurately. The experimental results demonstrate that MixSP decreased the overlap representation between upper-range and lower-range classes significantly while outperforming competitors on STS and zero-shot benchmarks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 5 Jun 2024.
Builds on
21 of 38 references.
References · 21
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
- 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
- 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
- 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
- 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
- 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
9 more
- 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
- 2020Universal Sentence Representation Learning with Conditional Masked Language Modelcited by 34
- 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
- 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
- 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20
- 2024Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignmentcited by 13
- 2022Differentiable Data Augmentation for Contrastive Sentence Representation Learningcited by 9
- 2020Efficient Sentence Embedding via Semantic Subspace Analysiscited by 4
- 2023An Efficient Self-Supervised Cross-View Training For Sentence Embeddingcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.