# Joint Language Semantic and Structure Embedding for Knowledge Graph Completion arXiv: https://arxiv.org/abs/2209.08721 ยท HTML: https://mlsanity.com/paper/2209.08721/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 59 references that are in the map. - First posted on arXiv 19 Sep 2022. ## Builds on (most cited first, 29 of 29) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019) - [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018) - [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019) - [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019) - [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019) - [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019) - [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014) - [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016) - [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019) - [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017) - [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020) - [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019) - [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019) - [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019) - [Low-Dimensional Hyperbolic Knowledge Graph Embeddings](https://mlsanity.com/paper/2005.00545/) (2020) - [A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network](https://mlsanity.com/paper/1712.02121/) (2017) - [Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction](https://mlsanity.com/paper/1911.09419/) (2019) - [Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs](https://mlsanity.com/paper/1906.01195/) (2019) - [Traversing Knowledge Graphs in Vector Space](https://mlsanity.com/paper/1506.01094/) (2015) - [Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations](https://mlsanity.com/paper/2110.02834/) (2021) - [TransG : A Generative Mixture Model for Knowledge Graph Embedding](https://mlsanity.com/paper/1509.05488/) (2015) - [Duality-Induced Regularizer for Tensor Factorization Based Knowledge Graph Completion](https://mlsanity.com/paper/2011.05816/) (2020) - [DOLORES: Deep Contextualized Knowledge Graph Embeddings](https://mlsanity.com/paper/1811.00147/) (2018) - [NePTuNe: Neural Powered Tucker Network for Knowledge Graph Completion](https://mlsanity.com/paper/2104.07824/) (2021) ## Built on by (most cited first, 15 of 15) - [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023) - [Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends](https://mlsanity.com/paper/2409.14457/) (2024) - [Benchmarking Language Models for Code Syntax Understanding](https://mlsanity.com/paper/2210.14473/) (2022) - [Large Language Model Enhanced Knowledge Representation Learning: A Survey](https://mlsanity.com/paper/2407.00936/) (2024) - [Large language models as oracles for instantiating ontologies with domain-specific knowledge](https://mlsanity.com/paper/2404.04108/) (2024) - [PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion](https://mlsanity.com/paper/2210.13715/) (2022) - [HaSa: Hardness and Structure-Aware Contrastive Knowledge Graph Embedding](https://mlsanity.com/paper/2305.10563/) (2023) - [Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model](https://mlsanity.com/paper/2312.01837/) (2023) - [IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models](https://mlsanity.com/paper/2210.14128/) (2022) - [Knowledge Graph Completion using Structural and Textual Embeddings](https://mlsanity.com/paper/2404.16206/) (2024) - [KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models](https://mlsanity.com/paper/2506.03576/) (2025) - [Flow-Modulated Scoring for Semantic-Aware Knowledge Graph Completion](https://mlsanity.com/paper/2506.23137/) (2025) - [FLAME: Empowering Frozen LLMs for Knowledge Graph Completion](https://mlsanity.com/paper/2408.06787/) (2024) - [Diversified and Adaptive Negative Sampling on Knowledge Graphs](https://mlsanity.com/paper/2410.07592/) (2024) - [BioGraphFusion: Graph Knowledge Embedding for Biological Completion and Reasoning](https://mlsanity.com/paper/2507.14468/) (2025) ## Abstract (arXiv) The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike previous approaches that rely on either the structures or semantics of the knowledge graphs, we propose to jointly embed the semantics in the natural language description of the knowledge triplets with their structure information. Our method embeds knowledge graphs for the completion task via fine-tuning pre-trained language models with respect to a probabilistic structured loss, where the forward pass of the language models captures semantics and the loss reconstructs structures. Our extensive experiments on a variety of knowledge graph benchmarks have demonstrated the state-of-the-art performance of our method. We also show that our method can significantly improve the performance in a low-resource regime, thanks to the better use of semantics. The code and datasets are available at https://github.com/pkusjh/LASS.