# Enriching BERT with Knowledge Graph Embeddings for Document Classification

arXiv: https://arxiv.org/abs/1909.08402 · HTML: https://mlsanity.com/paper/1909.08402/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2019.

## Built on by (most cited first, 22 of 22)

- [Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey](https://mlsanity.com/paper/2110.08455/) (2021)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media](https://mlsanity.com/paper/2204.04046/) (2022)
- [Graph Neural Networks for Text Classification: A Survey](https://mlsanity.com/paper/2304.11534/) (2023)
- [Knowledge Graph informed Fake News Classification via Heterogeneous Representation Ensembles](https://mlsanity.com/paper/2110.10457/) (2021)
- [Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognition](https://mlsanity.com/paper/2108.00194/) (2021)
- [Few-shot Learning for Cross-Target Stance Detection by Aggregating Multimodal Embeddings](https://mlsanity.com/paper/2301.04535/) (2023)
- [Leveraging Large Language Models for Semantic Query Processing in a Scholarly Knowledge Graph](https://mlsanity.com/paper/2405.15374/) (2024)
- [Evaluating Document Representations for Content-based Legal Literature Recommendations](https://mlsanity.com/paper/2104.13841/) (2021)
- [Applying Automatic Text Summarization for Fake News Detection](https://mlsanity.com/paper/2204.01841/) (2022)
- [KG-MTT-BERT: Knowledge Graph Enhanced BERT for Multi-Type Medical Text Classification](https://mlsanity.com/paper/2210.03970/) (2022)
- [Focusing Knowledge-based Graph Argument Mining via Topic Modeling](https://mlsanity.com/paper/2102.02086/) (2021)
- [AutoML-guided Fusion of Entity and LLM-based Representations for Document Classification](https://mlsanity.com/paper/2408.09794/) (2024)
- [A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.07792/) (2021)
- [Enriching language models with graph-based context information to better understand textual data](https://mlsanity.com/paper/2305.11070/) (2023)
- [SocialPET: Socially Informed Pattern Exploiting Training for Few-Shot Stance Detection in Social Media](https://mlsanity.com/paper/2403.05216/) (2024)
- [Enriched BERT Embeddings for Scholarly Publication Classification](https://mlsanity.com/paper/2405.04136/) (2024)
- [MessageNet: Message Classification using Natural Language Processing and Meta-data](https://mlsanity.com/paper/2301.01808/) (2023)
- [Leveraging Knowledge Graph Embeddings to Enhance Contextual Representations for Relation Extraction](https://mlsanity.com/paper/2306.04203/) (2023)
- [Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs](https://mlsanity.com/paper/2309.16357/) (2023)
- [Relational Graph Convolutional Networks for Sentiment Analysis](https://mlsanity.com/paper/2404.13079/) (2024)
- [FuDoBa: Fusing Document and Knowledge Graph-based Representations with Bayesian Optimisation](https://mlsanity.com/paper/2507.06622/) (2025)

## Abstract (arXiv)

In this paper, we focus on the classification of books using short descriptive texts (cover blurbs) and additional metadata. Building upon BERT, a deep neural language model, we demonstrate how to combine text representations with metadata and knowledge graph embeddings, which encode author information. Compared to the standard BERT approach we achieve considerably better results for the classification task. For a more coarse-grained classification using eight labels we achieve an F1- score of 87.20, while a detailed classification using 343 labels yields an F1-score of 64.70. We make the source code and trained models of our experiments publicly available