# MED-SE: Medical Entity Definition-based Sentence Embedding

arXiv: https://arxiv.org/abs/2212.04734 · HTML: https://mlsanity.com/paper/2212.04734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 9 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021)
- [On the Sentence Embeddings from Pre-trained Language Models](https://mlsanity.com/paper/2011.05864/) (2020)
- [WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach](https://mlsanity.com/paper/2104.01767/) (2021)
- [BioSentVec: creating sentence embeddings for biomedical texts](https://mlsanity.com/paper/1810.09302/) (2018)
- [MedSTS: A Resource for Clinical Semantic Textual Similarity](https://mlsanity.com/paper/1808.09397/) (2018)
- [EASE: Entity-Aware Contrastive Learning of Sentence Embedding](https://mlsanity.com/paper/2205.04260/) (2022)
- [CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes](https://mlsanity.com/paper/2106.02524/) (2021)
- [MedCAT -- Medical Concept Annotation Tool](https://mlsanity.com/paper/1912.10166/) (2019)

## Abstract (arXiv)

We propose Medical Entity Definition-based Sentence Embedding (MED-SE), a novel unsupervised contrastive learning framework designed for clinical texts, which exploits the definitions of medical entities. To this end, we conduct an extensive analysis of multiple sentence embedding techniques in clinical semantic textual similarity (STS) settings. In the entity-centric setting that we have designed, MED-SE achieves significantly better performance, while the existing unsupervised methods including SimCSE show degraded performance. Our experiments elucidate the inherent discrepancies between the general- and clinical-domain texts, and suggest that entity-centric contrastive approaches may help bridge this gap and lead to a better representation of clinical sentences.