mlsanityData to 14 Oct 2026

SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts

arXiv 2507.13105 ↗First posted 17 Jul 2025cs.CL cs.IR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce SemCSE, an unsupervised method for learning semantic embeddings of scientific texts. Building on recent advances in contrastive learning for text embeddings, our approach leverages LLM-generated summaries of scientific abstracts to train a model that positions semantically related summaries closer together in the embedding space. This resulting objective ensures that the model captures the true semantic content of a text, in contrast to traditional citation-based approaches that do not necessarily reflect semantic similarity. To validate this, we propose a novel benchmark designed to assess a model's ability to understand and encode the semantic content of scientific texts, demonstrating that our method enforces a stronger semantic separation within the embedding space. Additionally, we evaluate SemCSE on the comprehensive SciRepEval benchmark for scientific text embeddings, where it achieves state-of-the-art performance among models of its size, thus highlighting the benefits of a semantically focused training approach.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

28 of 44 references.

References · 28

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  5. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  8. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  9. 2014Deep metric learning using Triplet networkcited by 422
  10. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
  11. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  12. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
16 more
  1. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
  2. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 104
  3. 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
  4. 2022Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddingscited by 75
  5. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  6. 2018Content-Based Citation Recommendationcited by 43
  7. 2021Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similaritycited by 35
  8. 2024GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddingscited by 15
  9. 2021Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrievalcited by 14
  10. 2025Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performancecited by 12
  11. 2024Little Giants: Synthesizing High-Quality Embedding Data at Scalecited by 11
  12. 2019A Scalable Hybrid Research Paper Recommender System for Microsoft Academiccited by 10
  13. 2016Science Concierge: A fast content-based recommendation system for scientific publicationscited by 4
  14. 2025Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Themcited by 4
  15. 2021Document Embedding for Scientific Articles: Efficacy of Word Embeddings vs TFIDFcited by 3
  16. 2024Preserving Multilingual Quality While Tuning Query Encoder on English Onlycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddingscited by 5
  2. 2025SemCSE-Multi: Multifaceted and Decodable Embeddings for Aspect-Specific and Interpretable Scientific Domain Mappingcited by 1
  3. 2026Asymmetric Within-Document Predictive Learning for Scientific Document Representationcited by 0
  4. 2026Same Problem, Different Field: Cross-Domain Solution Import via Domain-Stripped Computational Fingerprintscited by 0