Unified Representation of Genomic and Biomedical Concepts through Multi-Task, Multi-Source Contrastive Learning
Not on the map. Why?
Abstract (arXiv)
We introduce GENomic Encoding REpresentation with Language Model (GENEREL), a framework designed to bridge genetic and biomedical knowledge bases. What sets GENEREL apart is its ability to fine-tune language models to infuse biological knowledge behind clinical concepts such as diseases and medications. This fine-tuning enables the model to capture complex biomedical relationships more effectively, enriching the understanding of how genomic data connects to clinical outcomes. By constructing a unified embedding space for biomedical concepts and a wide range of common SNPs from sources such as patient-level data, biomedical knowledge graphs, and GWAS summaries, GENEREL aligns the embeddings of SNPs and clinical concepts through multi-task contrastive learning. This allows the model to adapt to diverse natural language representations of biomedical concepts while bypassing the limitations of traditional code mapping systems across different data sources. Our experiments demonstrate GENEREL's ability to effectively capture the nuanced relationships between SNPs and clinical concepts. GENEREL also emerges to discern the degree of relatedness, potentially allowing for a more refined identification of concepts. This pioneering approach in constructing a unified embedding system for both SNPs and biomedical concepts enhances the potential for data integration and discovery in biomedical research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 56 references that are in the map. First posted on arXiv 14 Oct 2024.
Builds on
24 of 56 references.
References · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023GPT-4 Technical Reportcited by 18,396
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
- 2019Publicly Available Clinical BERT Embeddingscited by 1,002
- 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 932
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
- 2020A Survey on Contrastive Self-supervised Learningcited by 554
- 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
- 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
12 more
- 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 237
- 2018SimplE Embedding for Link Prediction in Knowledge Graphscited by 195
- 2017Predicting multicellular function through multi-layer tissue networkscited by 138
- 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
- 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
- 2021Graph Representation Learning in Biomedicinecited by 80
- 2019Graph Representation Learning: A Surveycited by 75
- 2016Multi-layer Representation Learning for Medical Conceptscited by 69
- 2020CODER: Knowledge infused cross-lingual medical term embedding for term normalizationcited by 55
- 2020COMETA: A Corpus for Medical Entity Linking in the Social Mediacited by 50
- 2022Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuningcited by 35
- 2021Knowledge-Rich Self-Supervision for Biomedical Entity Linkingcited by 31
Built on by
Not cited yet.