# SEMMA: A Semantic Aware Knowledge Graph Foundation Model

arXiv: https://arxiv.org/abs/2505.20422 · HTML: https://mlsanity.com/paper/2505.20422/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 72 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 30 of 39)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning](https://mlsanity.com/paper/1707.06690/) (2017)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)
- [jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://mlsanity.com/paper/2409.10173/) (2024)
- [Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2010.11465/) (2020)
- [CoDEx: A Comprehensive Knowledge Graph Completion Benchmark](https://mlsanity.com/paper/2009.07810/) (2020)
- [Position: Graph Foundation Models are Already Here](https://mlsanity.com/paper/2402.02216/) (2024)
- [End-to-End Neural Entity Linking](https://mlsanity.com/paper/1808.07699/) (2018)
- [Inductive Entity Representations from Text via Link Prediction](https://mlsanity.com/paper/2010.03496/) (2020)
- [Commonsense Knowledge Base Completion with Structural and Semantic Context](https://mlsanity.com/paper/1910.02915/) (2019)
- [InGram: Inductive Knowledge Graph Embedding via Relation Graphs](https://mlsanity.com/paper/2305.19987/) (2023)
- [Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations](https://mlsanity.com/paper/2110.02834/) (2021)
- [Improving Knowledge Graph Embedding Using Simple Constraints](https://mlsanity.com/paper/1805.02408/) (2018)
- [Fusing Context Into Knowledge Graph for Commonsense Question Answering](https://mlsanity.com/paper/2012.04808/) (2020)
- [A Survey on Temporal Knowledge Graph: Representation Learning and Applications](https://mlsanity.com/paper/2403.04782/) (2024)
- [Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph](https://mlsanity.com/paper/2010.01899/) (2020)
- [An Open Challenge for Inductive Link Prediction on Knowledge Graphs](https://mlsanity.com/paper/2203.01520/) (2022)
- [A Multi-Task Perspective for Link Prediction with New Relation Types and Nodes](https://mlsanity.com/paper/2307.06046/) (2023)
- [TRIX: A More Expressive Model for Zero-shot Domain Transfer in Knowledge Graphs](https://mlsanity.com/paper/2502.19512/) (2025)
- [How Expressive are Knowledge Graph Foundation Models?](https://mlsanity.com/paper/2502.13339/) (2025)
- [Double Equivariance for Inductive Link Prediction for Both New Nodes and New Relation Types](https://mlsanity.com/paper/2302.01313/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models](https://mlsanity.com/paper/2508.20583/) (2025)
- [Half a Link can Be Enough to Predict a Whole Link: Understanding Generalization in Knowledge Graph Foundation Models](https://mlsanity.com/paper/2606.18001/) (2026)
- [OpenGloss: A Synthetic Encyclopedic Dictionary and Semantic Knowledge Graph](https://mlsanity.com/paper/2511.18622/) (2025)
- [UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG](https://mlsanity.com/paper/2603.28773/) (2026)
- [Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling](https://mlsanity.com/paper/2605.27023/) (2026)
- [Graph Evidence Is Not Enough: Diagnosing Native Decoder Use in Graph-Augmented LLMs](https://mlsanity.com/paper/2608.30437/) (2026)
- [Reification as a Transferable Vocabulary: Zero-Shot Link Prediction with Vanilla GNNs](https://mlsanity.com/paper/2609.11347/) (2026)
- [MERIT: Scaling KG Foundation Models with Relation Transformers](https://mlsanity.com/paper/2610.15638/) (2026)

## Abstract (arXiv)

Knowledge Graph Foundation Models (KGFMs) have shown promise in enabling zero-shot reasoning over unseen graphs by learning transferable patterns. However, most existing KGFMs rely solely on graph structure, overlooking the rich semantic signals encoded in textual attributes. We introduce SEMMA, a dual-module KGFM that systematically integrates transferable textual semantics alongside structure. SEMMA leverages Large Language Models (LLMs) to enrich relation identifiers, generating semantic embeddings that subsequently form a textual relation graph, which is fused with the structural component. Across 54 diverse KGs, SEMMA outperforms purely structural baselines like ULTRA in fully inductive link prediction. Crucially, we show that in more challenging generalization settings, where the test-time relation vocabulary is entirely unseen, structural methods collapse while SEMMA is 2x more effective. Our findings demonstrate that textual semantics are critical for generalization in settings where structure alone fails, highlighting the need for foundation models that unify structural and linguistic signals in knowledge reasoning.