mlsanityData to 7 Oct 2026

Joint Language Semantic and Structure Embedding for Knowledge Graph Completion

arXiv 2209.08721 ↗First posted 19 Sep 2022cs.CL cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike previous approaches that rely on either the structures or semantics of the knowledge graphs, we propose to jointly embed the semantics in the natural language description of the knowledge triplets with their structure information. Our method embeds knowledge graphs for the completion task via fine-tuning pre-trained language models with respect to a probabilistic structured loss, where the forward pass of the language models captures semantics and the loss reconstructs structures. Our extensive experiments on a variety of knowledge graph benchmarks have demonstrated the state-of-the-art performance of our method. We also show that our method can significantly improve the performance in a low-resource regime, thanks to the better use of semantics. The code and datasets are available at https://github.com/pkusjh/LASS.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 29 of its 59 references that are in the map. First posted on arXiv 19 Sep 2022.

Builds on

29 of 59 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2018Deep contextualized word representationscited by 2,285
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  12. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
17 more
  1. 2016Complex Embeddings for Simple Link Predictioncited by 897
  2. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 893
  3. 2017Convolutional 2D Knowledge Graph Embeddingscited by 729
  4. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  5. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  6. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 323
  7. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  8. 2020Low-Dimensional Hyperbolic Knowledge Graph Embeddingscited by 195
  9. 2017A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Networkcited by 143
  10. 2019Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Predictioncited by 133
  11. 2019Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphscited by 125
  12. 2015Traversing Knowledge Graphs in Vector Spacecited by 51
  13. 2021Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representationscited by 37
  14. 2015TransG : A Generative Mixture Model for Knowledge Graph Embeddingcited by 28
  15. 2020Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completioncited by 28
  16. 2018DOLORES: Deep Contextualized Knowledge Graph Embeddingscited by 11
  17. 2021NePTuNe: Neural Powered Tucker Network for Knowledge Graph Completioncited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 3; 2024: 6; 2025: 3.

15 papers.

Sort:

By area:

  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  2. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  3. 2022Benchmarking Language Models for Code Syntax Understandingcited by 8
  4. 2024Large language models as oracles for instantiating ontologies with domain-specific knowledgecited by 6
  5. 2024Large Language Model Enhanced Knowledge Representation Learning: A Surveycited by 6
  6. 2022PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completioncited by 3
  7. 2023HaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embeddingcited by 3
  8. 2023Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Modelcited by 3
  9. 2022IELM: An Open Information Extraction Benchmark for Pre-Trained Language Modelscited by 1
  10. 2024Knowledge Graph Completion using Structural and Textual Embeddingscited by 1
  11. 2025KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Modelscited by 1
  12. 2025Flow-Modulated Scoring for Semantic-Aware Knowledge Graph Completioncited by 1
3 more
  1. 2024FLAME: Empowering Frozen LLMs for Knowledge Graph Completioncited by 0
  2. 2024Diversified and Adaptive Negative Sampling on Knowledge Graphscited by 0
  3. 2025BioGraphFusion: Graph Knowledge Embedding for Biological Completion and Reasoningcited by 0