# Graph-Augmented Cyclic Learning Framework for Similarity Estimation of Medical Clinical Notes

arXiv: https://arxiv.org/abs/2208.09437 · HTML: https://mlsanity.com/paper/2208.09437/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 19 Aug 2022.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://mlsanity.com/paper/1602.03483/) (2016)
- [MedSTS: A Resource for Clinical Semantic Textual Similarity](https://mlsanity.com/paper/1808.09397/) (2018)

## Abstract (arXiv)

Semantic textual similarity (STS) in the clinical domain helps improve diagnostic efficiency and produce concise texts for downstream data mining tasks. However, given the high degree of domain knowledge involved in clinic text, it remains challenging for general language models to infer implicit medical relationships behind clinical sentences and output similarities correctly. In this paper, we present a graph-augmented cyclic learning framework for similarity estimation in the clinical domain. The framework can be conveniently implemented on a state-of-art backbone language model, and improve its performance by leveraging domain knowledge through co-training with an auxiliary graph convolution network (GCN) based network. We report the success of introducing domain knowledge in GCN and the co-training framework by improving the Bio-clinical BERT baseline by 16.3% and 27.9%, respectively.