mlsanityData to 7 Oct 2026

Leveraging Knowledge Graph Embeddings to Enhance Contextual Representations for Relation Extraction

arXiv 2306.04203 ↗First posted 7 Jun 2023cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Relation extraction task is a crucial and challenging aspect of Natural Language Processing. Several methods have surfaced as of late, exhibiting notable performance in addressing the task; however, most of these approaches rely on vast amounts of data from large-scale knowledge graphs or language models pretrained on voluminous corpora. In this paper, we hone in on the effective utilization of solely the knowledge supplied by a corpus to create a high-performing model. Our objective is to showcase that by leveraging the hierarchical structure and relational distribution of entities within a corpus without introducing external knowledge, a relation extraction model can achieve significantly enhanced performance. We therefore proposed a relation extraction approach based on the incorporation of pretrained knowledge graph embeddings at the corpus scale into the sentence-level contextual representation. We conducted a series of experiments which revealed promising and very interesting results for our proposed approach.The obtained results demonstrated an outperformance of our method compared to context-based relation extraction models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

18 of 28 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  4. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  5. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  6. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  7. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  8. 2022Galactica: A Large Language Model for Sciencecited by 736
  9. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  10. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
  11. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  12. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
6 more
  1. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  2. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  3. 2017Knowledge Graph Completion via Complex Tensor Factorizationcited by 80
  4. 2021An Improved Baseline for Sentence-level Relation Extractioncited by 80
  5. 2019Enriching BERT with Knowledge Graph Embeddings for Document Classificationcited by 22
  6. 2022Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasetscited by 11

Built on by

Not cited yet.