# Transformer-based Entity Typing in Knowledge Graphs

arXiv: https://arxiv.org/abs/2210.11151 · HTML: https://mlsanity.com/paper/2210.11151/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 20 Oct 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [HittER: Hierarchical Transformers for Knowledge Graph Embeddings](https://mlsanity.com/paper/2008.12813/) (2020)
- [Modeling Fine-Grained Entity Types with Box Embeddings](https://mlsanity.com/paper/2101.00345/) (2021)
- [Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling](https://mlsanity.com/paper/2106.01040/) (2021)
- [CoKE: Contextualized Knowledge Graph Embedding](https://mlsanity.com/paper/1911.02168/) (2019)
- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [DA-Transformer: Distance-aware Transformer](https://mlsanity.com/paper/2010.06925/) (2020)
- [Fine-grained Entity Typing via Label Reasoning](https://mlsanity.com/paper/2109.05744/) (2021)
- [Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types](https://mlsanity.com/paper/1802.06842/) (2018)
- [Connecting Embeddings for Knowledge Graph Entity Typing](https://mlsanity.com/paper/2007.10873/) (2020)
- [Type-aware Embeddings for Multi-Hop Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2205.00782/) (2022)
- [Context-aware Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2109.07990/) (2021)
- [CORE: A Knowledge Graph Entity Type Prediction Method via Complex Space Regression and Embedding](https://mlsanity.com/paper/2112.10067/) (2021)

## Built on by (most cited first, 13 of 13)

- [TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness](https://mlsanity.com/paper/2402.12545/) (2024)
- [Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer](https://mlsanity.com/paper/2303.03922/) (2023)
- [Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning](https://mlsanity.com/paper/2307.03591/) (2023)
- [Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph](https://mlsanity.com/paper/2405.00352/) (2024)
- [Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2310.06365/) (2023)
- [The Integration of Semantic and Structural Knowledge in Knowledge Graph Entity Typing](https://mlsanity.com/paper/2404.08313/) (2024)
- [HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion](https://mlsanity.com/paper/2308.06512/) (2023)
- [AsyncET: Asynchronous Learning for Knowledge Graph Entity Typing with Auxiliary Relations](https://mlsanity.com/paper/2308.16055/) (2023)
- [Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs](https://mlsanity.com/paper/2310.12008/) (2023)
- [HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation](https://mlsanity.com/paper/2404.09848/) (2024)
- [Are We Wasting Time? A Fast, Accurate Performance Evaluation Framework for Knowledge Graph Link Predictors](https://mlsanity.com/paper/2402.00053/) (2024)
- [COTET: Cross-view Optimal Transport for Knowledge Graph Entity Typing](https://mlsanity.com/paper/2405.13602/) (2024)
- [Evaluating and Improving Graph to Text Generation with Large Language Models](https://mlsanity.com/paper/2501.14497/) (2025)

## Abstract (arXiv)

We investigate the knowledge graph entity typing task which aims at inferring plausible entity types. In this paper, we propose a novel Transformer-based Entity Typing (TET) approach, effectively encoding the content of neighbors of an entity. More precisely, TET is composed of three different mechanisms: a local transformer allowing to infer missing types of an entity by independently encoding the information provided by each of its neighbors; a global transformer aggregating the information of all neighbors of an entity into a single long sequence to reason about more complex entity types; and a context transformer integrating neighbors content based on their contribution to the type inference through information exchange between neighbor pairs. Furthermore, TET uses information about class membership of types to semantically strengthen the representation of an entity. Experiments on two real-world datasets demonstrate the superior performance of TET compared to the state-of-the-art.