# A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Models

arXiv: https://arxiv.org/abs/2402.10779 · HTML: https://mlsanity.com/paper/2402.10779/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [TuckER: Tensor Factorization for Knowledge Graph Completion](https://mlsanity.com/paper/1901.09590/) (2019)
- [Language is All a Graph Needs](https://mlsanity.com/paper/2308.07134/) (2023)
- [SimplE Embedding for Link Prediction in Knowledge Graphs](https://mlsanity.com/paper/1802.04868/) (2018)
- [Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning](https://mlsanity.com/paper/1711.05851/) (2017)
- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [Knowledge Graph Embedding for Link Prediction: A Comparative Analysis](https://mlsanity.com/paper/2002.00819/) (2020)
- [Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models](https://mlsanity.com/paper/2401.00625/) (2024)
- [Making Large Language Models Perform Better in Knowledge Graph Completion](https://mlsanity.com/paper/2310.06671/) (2023)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [OntoZSL: Ontology-enhanced Zero-shot Learning](https://mlsanity.com/paper/2102.07339/) (2021)
- [Exploring Large Language Models for Knowledge Graph Completion](https://mlsanity.com/paper/2308.13916/) (2023)
- [Path Neural Networks: Expressive and Accurate Graph Neural Networks](https://mlsanity.com/paper/2306.05955/) (2023)
- [Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs](https://mlsanity.com/paper/2001.02332/) (2020)
- [Collaborative Policy Learning for Open Knowledge Graph Reasoning](https://mlsanity.com/paper/1909.00230/) (2019)
- [Degree-Aware Alignment for Entities in Tail](https://mlsanity.com/paper/2005.12132/) (2020)
- [Semantic Structure based Query Graph Prediction for Question Answering over Knowledge Graph](https://mlsanity.com/paper/2204.10194/) (2022)
- [A Hierarchical N-Gram Framework for Zero-Shot Link Prediction](https://mlsanity.com/paper/2204.10293/) (2022)
- [Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Tracking](https://mlsanity.com/paper/2304.13854/) (2023)
- [Multi-Fusion Chinese WordNet (MCW) : Compound of Machine Learning and Manual Correction](https://mlsanity.com/paper/2002.01761/) (2020)

## Built on by (most cited first, 3 of 3)

- [BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine](https://mlsanity.com/paper/2405.00465/) (2024)
- [CancerLLM: A Large Language Model in Cancer Domain](https://mlsanity.com/paper/2406.10459/) (2024)
- [RIVO: Lightweight Adaptive Rerouting for Deep Research Agents](https://mlsanity.com/paper/2610.15910/) (2026)

## Abstract (arXiv)

Zero-shot link prediction (ZSLP) on knowledge graphs aims at automatically identifying relations between given entities. Existing methods primarily employ auxiliary information to predict tail entity given head entity and its relation, yet face challenges due to the occasional unavailability of such detailed information and the inherent simplicity of predicting tail entities based on semantic similarities. Even though Large Language Models (LLMs) offer a promising solution to predict unobserved relations between the head and tail entity in a zero-shot manner, their performance is still restricted due to the inability to leverage all the (exponentially many) paths' information between two entities, which are critical in collectively indicating their relation types. To address this, in this work, we introduce a Condensed Transition Graph Framework for Zero-Shot Link Prediction (CTLP), which encodes all the paths' information in linear time complexity to predict unseen relations between entities, attaining both efficiency and information preservation. Specifically, we design a condensed transition graph encoder with theoretical guarantees on its coverage, expressiveness, and efficiency. It is learned by a transition graph contrastive learning strategy. Subsequently, we design a soft instruction tuning to learn and map the all-path embedding to the input of LLMs. Experimental results show that our proposed CTLP method achieves state-of-the-art performance on three standard ZSLP datasets