mlsanityData to 7 Oct 2026

BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)

arXiv 2210.11892 ↗First posted 21 Oct 2022cs.CL cs.IRcited by 20

Not on the map. Why?

Abstract (arXiv)

This work introduces BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts. State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations. BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (MayoSRS).

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

20 of 31 references.

References · 20

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  5. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  6. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  9. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  10. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  11. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  12. 2019MedMentions: A Large Biomedical Corpus Annotated with UMLS Conceptscited by 79
8 more
  1. 2020Biomedical Entity Representations with Synonym Marginalizationcited by 59
  2. 2021Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processingcited by 57
  3. 2018MedSTS: A Resource for Clinical Semantic Textual Similaritycited by 39
  4. 2021Knowledge-Rich Self-Supervision for Biomedical Entity Linkingcited by 31
  5. 2021A Hybrid Approach to Measure Semantic Relatedness in Biomedical Conceptscited by 5
  6. 2021Biomedical Entity Linking with Contrastive Context Matchingcited by 5
  7. 2020Biomedical Concept Relatedness -- A large EHR-based benchmarkcited by 4
  8. 2021Neural sentence embedding models for semantic similarity estimation in the biomedical domaincited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 6; 2025: 3; 2026: 6.

20 papers.

Sort:

By area:

  1. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  2. 2025An Agentic System for Rare Disease Diagnosis with Traceable Reasoningcited by 50
  3. 2024Towards Evaluating and Building Versatile Large Language Models for Medicinecited by 38
  4. 2023Extreme Multi-Label Skill Extraction Training using Large Language Modelscited by 17
  5. 2023BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insightscited by 7
  6. 2023Automatic Glossary of Clinical Terminology: a Large-Scale Dictionary of Biomedical Definitions Generated from Ontological Knowledgecited by 5
  7. 2024In-Context Learning for Extreme Multi-Label Classificationcited by 4
  8. 2023Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contextscited by 2
  9. 2024Evaluation of Language Models in the Medical Context Under Resource-Constrained Settingscited by 1
  10. 2024A Natural Language Processing Approach to Support Biomedical Data Harmonization: Leveraging Large Language Modelscited by 1
  11. 2026A Vision-language Framework for Comparative Reasoning in Radiologycited by 1
  12. 2026When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Imagescited by 1
8 more
  1. 2023Detecting Idiomatic Multiword Expressions in Clinical Terminology using Definition-Based Representation Learningcited by 0
  2. 2024From Text to Treatment Effects: A Meta-Learning Approach to Handling Text-Based Confoundingcited by 0
  3. 2025Semantic Alignment of Unimodal Medical Text and Vision Representationscited by 0
  4. 2025InterFeat: A Pipeline for Finding Interesting Scientific Featurescited by 0
  5. 2026Language corpora for the Dutch medical domaincited by 0
  6. 2026Transfer Learning for FHIR Questionnaire Terminology Bindingcited by 0
  7. 2026OntologyAligner: Ontology-Aligned Retrieval and Hierarchy-Guided Large Language Model Reranking for Biomedical Ontology Normalizationcited by 0
  8. 2026EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Eventscited by 0