BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts
Not on the map. Why?
Abstract (arXiv)
State-of-the-art models for relation extraction (RE) in the biomedical domain consider finetuning BioBERT using classification, but they may suffer from the anisotropy problem. Contrastive learning methods can reduce this anisotropy phenomena, and also help to avoid class collapse in any classification problem. In the present paper, a new training method called biological non-contrastive relation extraction (BioNCERE) is introduced for relation extraction without using any named entity labels for training to reduce annotation costs. BioNCERE uses transfer learning and non-contrastive learning to avoid full or dimensional collapse as well as bypass overfitting. It resolves RE in three stages by leveraging transfer learning two times. By freezing the weights learned in previous stages in the proposed pipeline and by leveraging non-contrastive learning in the second stage, the model predicts relations without any knowledge of named entities. Experiments have been done on SemMedDB that are almost similar to State-of-the-art performance on RE without using the information of named entities.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 31 Oct 2024.
Builds on
28 of 36 references.
References · 28
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2020Supervised Contrastive Learningcited by 2,803
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2020What Makes for Good Views for Contrastive Learning?cited by 843
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
16 more
- 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
- 2022How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learningcited by 65
- 2022Understanding Collapse in Non-Contrastive Siamese Representation Learningcited by 42
- 2022Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learningcited by 40
- 2023Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Trainingcited by 39
- 2022HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extractioncited by 32
- 2022ContraCLM: Contrastive Learning For Causal Language Modelcited by 18
- 2022Addressing Token Uniformity in Transformers via Singular Value Transformationcited by 14
- 2020Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approachcited by 12
- 2021Improving BERT Model Using Contrastive Learning for Biomedical Relation Extractioncited by 10
- 2022Siamese Prototypical Contrastive Learningcited by 10
- 2023Mini-Batch Optimization of Contrastive Losscited by 8
- 2023Guarding Barlow Twins Against Overfitting with Mixed Samplescited by 7
- 2021Cluster Analysis with Deep Embeddings and Contrastive Learningcited by 3
- 2022Contrastive Training Improves Zero-Shot Classification of Semi-structured Documentscited by 3
- 2023Siamese Representation Learning for Unsupervised Relation Extractioncited by 1
Built on by
Not cited yet.