# AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge

arXiv: https://arxiv.org/abs/2307.07851 · HTML: https://mlsanity.com/paper/2307.07851/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [An efficient framework for learning sentence representations](https://mlsanity.com/paper/1803.02893/) (2018)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [A Unified Generative Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2106.04300/) (2021)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://mlsanity.com/paper/1602.03483/) (2016)
- [Aspect Based Sentiment Analysis with Gated Convolutional Networks](https://mlsanity.com/paper/1805.07043/) (2018)
- [A Decade of Knowledge Graphs in Natural Language Processing: A Survey](https://mlsanity.com/paper/2210.00105/) (2022)
- [Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches](https://mlsanity.com/paper/2211.16285/) (2022)
- [Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension](https://mlsanity.com/paper/2005.08056/) (2020)
- [Aspect-based Document Similarity for Research Papers](https://mlsanity.com/paper/2010.06395/) (2020)
- [PatternRank: Leveraging Pretrained Language Models and Part of Speech for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2210.05245/) (2022)
- [Specialized Document Embeddings for Aspect-based Similarity of Research Papers](https://mlsanity.com/paper/2203.14541/) (2022)
- [BiSyn-GAT+: Bi-Syntax Aware Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2204.03117/) (2022)
- [Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles](https://mlsanity.com/paper/2003.09881/) (2020)
- [Lbl2Vec: An Embedding-Based Approach for Unsupervised Document Retrieval on Predefined Topics](https://mlsanity.com/paper/2210.06023/) (2022)
- [Efficient Domain Adaptation of Sentence Embeddings Using Adapters](https://mlsanity.com/paper/2307.03104/) (2023)

## Built on by (most cited first, 5 of 5)

- [Interpretable Text Embeddings and Text Similarity Explanation: A Survey](https://mlsanity.com/paper/2502.14862/) (2025)
- [Exploring the Landscape of Natural Language Processing Research](https://mlsanity.com/paper/2307.10652/) (2023)
- [Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey](https://mlsanity.com/paper/2601.03270/) (2025)
- [The Critical Role of Aspects in Measuring Document Similarity](https://mlsanity.com/paper/2601.03435/) (2026)
- [Transforming OPACs into Intelligent Discovery Systems: An AI-Powered, Knowledge Graph-Driven Smart OPAC for Digital Libraries](https://mlsanity.com/paper/2604.01262/) (2026)

## Abstract (arXiv)

Generic sentence embeddings provide a coarse-grained approximation of semantic textual similarity but ignore specific aspects that make texts similar. Conversely, aspect-based sentence embeddings provide similarities between texts based on certain predefined aspects. Thus, similarity predictions of texts are more targeted to specific requirements and more easily explainable. In this paper, we present AspectCSE, an approach for aspect-based contrastive learning of sentence embeddings. Results indicate that AspectCSE achieves an average improvement of 3.97% on information retrieval tasks across multiple aspects compared to the previous best results. We also propose using Wikidata knowledge graph properties to train models of multi-aspect sentence embeddings in which multiple specific aspects are simultaneously considered during similarity predictions. We demonstrate that multi-aspect embeddings outperform single-aspect embeddings on aspect-specific information retrieval tasks. Finally, we examine the aspect-based sentence embedding space and demonstrate that embeddings of semantically similar aspect labels are often close, even without explicit similarity training between different aspect labels.