mlsanityData to 13 Oct 2026

MedSTS: A Resource for Clinical Semantic Textual Similarity

arXiv 1808.09397 ↗First posted 28 Aug 2018cs.IRcited by 39

Not on the map. Why?

Abstract (arXiv)

The wide adoption of electronic health records (EHRs) has enabled a wide range of applications leveraging EHR data. However, the meaningful use of EHR data largely depends on our ability to efficiently extract and consolidate information embedded in clinical text where natural language processing (NLP) techniques are essential. Semantic textual similarity (STS) that measures the semantic similarity between text snippets plays a significant role in many NLP applications. In the general NLP domain, STS shared tasks have made available a huge collection of text snippet pairs with manual annotations in various domains. In the clinical domain, STS can enable us to detect and eliminate redundant information that may lead to a reduction in cognitive burden and an improvement in the clinical decision-making process. This paper elaborates our efforts to assemble a resource for STS in the medical domain, MedSTS. It consists of a total of 174,629 sentence pairs gathered from a clinical corpus at Mayo Clinic. A subset of MedSTS (MedSTS_ann) containing 1,068 sentence pairs was annotated by two medical experts with semantic similarity scores of 0-5 (low to high similarity). We further analyzed the medical concepts in the MedSTS corpus, and tested four STS systems on the MedSTS_ann corpus. In the future, we will organize a shared task by releasing the MedSTS_ann corpus to motivate the community to tackle the real world clinical problems.

Key facts. Cited by 39 papers in the map (data through 13 Oct 2026). First posted on arXiv 28 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 8; 2022: 8; 2023: 11; 2024: 6; 2025: 4; 2026: 2.

39 papers.

Sort:

By area:

  1. 2022GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Recordscited by 238
  2. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  3. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  4. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  5. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 76
  6. 2021Memorization vs. Generalization: Quantifying Data Leakage in NLP Performance Evaluationcited by 69
  7. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  8. 2021GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domaincited by 36
  9. 2022BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)cited by 20
  10. 2023Transformers in Healthcare: A Surveycited by 17
  11. 2024Unveiling LLM Evaluation Focused on Metrics: Challenges and Solutionscited by 17
  12. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
27 more
  1. 2024Large Language Model Benchmarks in Medical Taskscited by 17
  2. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  3. 2021Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERTcited by 14
  4. 2021Estimating Redundancy in Clinical Textcited by 10
  5. 2023Collective Human Opinions in Semantic Textual Similaritycited by 8
  6. 2021Improving the robustness and accuracy of biomedical language models through adversarial trainingcited by 7
  7. 2023BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insightscited by 7
  8. 2022Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Wordscited by 6
  9. 2023Compositional Zero-Shot Domain Transfer with Text-to-Text Modelscited by 6
  10. 2024BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuningcited by 4
  11. 2022Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publicationscited by 3
  12. 2023Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contextscited by 2
  13. 2023Utilizing Semantic Textual Similarity for Clinical Survey Data Feature Selectioncited by 2
  14. 2023Rethinking STS and NLI in Large Language Modelscited by 2
  15. 2023SPICED: News Similarity Detection Dataset with Multiple Topics and Complexity Levelscited by 2
  16. 2025CaseReportBench: An LLM Benchmark Dataset for Dense Information Extraction in Clinical Case Reportscited by 2
  17. 2026Pantagruel: Unified Self-Supervised Encoders for French Text and Speechcited by 2
  18. 2026Diagnosable ColBERT: Debugging Late-Interaction Retrieval Models Using a Learned Latent Space as Referencecited by 2
  19. 2023DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applicationscited by 1
  20. 2024A Survey for Large Language Models in Biomedicinecited by 1
  21. 2025Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical textscited by 1
  22. 2021COVID-19: Comparative Analysis of Methods for Identifying Articles Related to Therapeutics and Vaccines without Using Labeled Datacited by 0
  23. 2022A reproducible experimental survey on biomedical sentence similarity: a string-based method sets the state of the artcited by 0
  24. 2022Graph-Augmented Cyclic Learning Framework for Similarity Estimation of Medical Clinical Notescited by 0
  25. 2022MED-SE: Medical Entity Definition-based Sentence Embeddingcited by 0
  26. 2025Search-Optimized Quantization in Biomedical Ontology Alignmentcited by 0
  27. 2025Evaluating Embedding Frameworks for Scientific Domaincited by 0