# Pretrained Transformers for Simple Question Answering over Knowledge Graphs

arXiv: https://arxiv.org/abs/2001.11985 · HTML: https://mlsanity.com/paper/2001.11985/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2020.

## Built on by (most cited first, 12 of 12)

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering](https://mlsanity.com/paper/2303.10368/) (2023)
- [Generative Relation Linking for Question Answering over Knowledge Bases](https://mlsanity.com/paper/2108.07337/) (2021)
- [Entity Retrieval for Answering Entity-Centric Questions](https://mlsanity.com/paper/2408.02795/) (2024)
- [Transformer-Based Models for Question Answering on COVID19](https://mlsanity.com/paper/2101.11432/) (2021)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [Investigating the Role of Feed-Forward Networks in Transformers Using Parallel Attention and Feed-Forward Net Design](https://mlsanity.com/paper/2305.13297/) (2023)
- [Core Building Blocks: Next Gen Geo Spatial GPT Application](https://mlsanity.com/paper/2310.11029/) (2023)
- [Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answering](https://mlsanity.com/paper/2409.15902/) (2024)
- [Knowledge Graph Question Answering via SPARQL Silhouette Generation](https://mlsanity.com/paper/2109.09475/) (2021)
- [Improving Customer Service Chatbots with Attention-based Transfer Learning](https://mlsanity.com/paper/2111.14621/) (2021)
- [Lexicalization Is All You Need: Examining the Impact of Lexical Knowledge in a Compositional QALD System](https://mlsanity.com/paper/2411.03906/) (2024)

## Abstract (arXiv)

Answering simple questions over knowledge graphs is a well-studied problem in question answering. Previous approaches for this task built on recurrent and convolutional neural network based architectures that use pretrained word embeddings. It was recently shown that finetuning pretrained transformer networks (e.g. BERT) can outperform previous approaches on various natural language processing tasks. In this work, we investigate how well BERT performs on SimpleQuestions and provide an evaluation of both BERT and BiLSTM-based models in datasparse scenarios.