mlsanityData to 7 Oct 2026

Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey

arXiv 2601.03270 ↗First posted 19 Dec 2025cs.CL cs.AI math.OCcited by 1

Not on the map. Why?

Abstract (arXiv)

Semantic Textual Similarity (STS) research has expanded rapidly since 2021, driven by advances in transformer architectures, contrastive learning, and domain-specific techniques. This survey reviews progress across six key areas: transformer-based models, contrastive learning, domain-focused solutions, multi-modal methods, graph-based approaches, and knowledge-enhanced techniques. Recent transformer models such as FarSSiBERT and DeBERTa-v3 have achieved remarkable accuracy, while contrastive methods like AspectCSE have established new benchmarks. Domain-adapted models, including CXR-BERT for medical texts and Financial-STS for finance, demonstrate how STS can be effectively customized for specialized fields. Moreover, multi-modal, graph-based, and knowledge-integrated models further enhance semantic understanding and representation. By organizing and analyzing these developments, the survey provides valuable insights into current methods, practical applications, and remaining challenges. It aims to guide researchers and practitioners alike in navigating rapid advancements, highlighting emerging trends and future opportunities in the evolving field of STS.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 81 of its 100 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

81 of 100 references.

References · 81

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  10. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  11. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  12. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
38 more
  1. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  2. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  3. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  4. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  5. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  6. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  7. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  8. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  9. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  10. 2021Entailment as Few-Shot Learnercited by 122
  11. 2019StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understandingcited by 108
  12. 2023AnglE-optimized Text Embeddingscited by 98
  13. 2023Scaling Sentence Embeddings with Large Language Modelscited by 83
  14. 2021How to Train BERT with an Academic Budgetcited by 80
  15. 2020RealFormer: Transformer Likes Residual Attentioncited by 62
  16. 2020A Statistical Framework for Low-bitwidth Training of Deep Neural Networkscited by 51
  17. 2020Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCOcited by 46
  18. 2023Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generationcited by 24
  19. 2024Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysiscited by 21
  20. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20
  21. 2024Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Methodcited by 18
  22. 2024Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narrativescited by 13
  23. 2023Text Encoders Lack Knowledge: Leveraging Generative LLMs for Domain-Specific Semantic Textual Similaritycited by 11
  24. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
  25. 2024DuSSS: Dual Semantic Similarity-Supervised Vision-Language Model for Semi-Supervised Medical Image Segmentationcited by 9
  26. 2022Adversarial Self-Attention for Language Understandingcited by 8
  27. 2023VacancySBERT: the approach for representation of titles and skills for semantic similarity search in the recruitment domaincited by 7
  28. 2024BMX: Entropy-weighted Similarity and Semantic-enhanced Lexical Searchcited by 7
  29. 2016Interpretable Semantic Textual Similarity: Finding and explaining differences between sentencescited by 5
  30. 2022ALBERT with Knowledge Graph Encoder Utilizing Semantic Similarity for Commonsense Question Answeringcited by 5
  31. 2023AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledgecited by 5
  32. 2023Improving Semantic Similarity Measure Within a Recommender System Based-on RDF Graphscited by 5
  33. 2023PEM: Representing Binary Program Semantics for Similarity Analysis via a Probabilistic Execution Modelcited by 5
  34. 2024Weakly-supervised Audio Separation via Bi-modal Semantic Similaritycited by 5
  35. 2024Semantic Similarity Score for Measuring Visual Similarity at Semantic Levelcited by 5
  36. 2024Pcc-tuning: Breaking the Contrastive Learning Ceiling in Semantic Textual Similaritycited by 5
  37. 2023Improving Text Semantic Similarity Modeling through a 3D Siamese Networkcited by 4
  38. 2024Semantic Textual Similarity Assessment in Chest X-ray Reports Using a Domain-Specific Cosine-Based Metriccited by 4

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Do General NLP Embeddings Capture Ontological Reasoning?cited by 0