# Leveraging Knowledge Bases in LSTMs for Improving Machine Reading

arXiv: https://arxiv.org/abs/1902.09091 · HTML: https://mlsanity.com/paper/1902.09091/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2019.

## Built on by (most cited first, 30 of 37)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [ChronoR: Rotation Based Temporal Knowledge Graph Embedding](https://mlsanity.com/paper/2103.10379/) (2021)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [Benchmarking Knowledge-Enhanced Commonsense Question Answering via Knowledge-to-Text Transformation](https://mlsanity.com/paper/2101.00760/) (2021)
- [Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference](https://mlsanity.com/paper/2105.13456/) (2021)
- [KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs](https://mlsanity.com/paper/2109.04223/) (2021)
- [Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering](https://mlsanity.com/paper/2105.11776/) (2021)
- [TempEL: Linking Dynamically Evolving and Newly Emerging Entities](https://mlsanity.com/paper/2302.02500/) (2023)
- [Injecting Knowledge Base Information into End-to-End Joint Entity and Relation Extraction and Coreference Resolution](https://mlsanity.com/paper/2107.02286/) (2021)
- [DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning](https://mlsanity.com/paper/2108.12370/) (2021)
- [SpherE: Expressive and Interpretable Knowledge Graph Embedding for Set Retrieval](https://mlsanity.com/paper/2404.19130/) (2024)
- [Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge](https://mlsanity.com/paper/2104.05216/) (2021)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation](https://mlsanity.com/paper/2311.09105/) (2023)
- [ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference](https://mlsanity.com/paper/2108.01589/) (2021)
- [Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision](https://mlsanity.com/paper/2107.08582/) (2021)
- [Fine-Grained Chemical Entity Typing with Multimodal Knowledge Representation](https://mlsanity.com/paper/2108.12899/) (2021)
- [A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning](https://mlsanity.com/paper/2308.07889/) (2023)
- [MCR-Net: A Multi-Step Co-Interactive Relation Network for Unanswerable Questions on Machine Reading Comprehension](https://mlsanity.com/paper/2103.04567/) (2021)
- [Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System](https://mlsanity.com/paper/2105.14313/) (2021)
- [Sound Logical Explanations for Mean Aggregation Graph Neural Networks](https://mlsanity.com/paper/2511.11593/) (2025)
- [SensPick: Sense Picking for Word Sense Disambiguation](https://mlsanity.com/paper/2102.05260/) (2021)
- [Incorporating Connections Beyond Knowledge Embeddings: A Plug-and-Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension](https://mlsanity.com/paper/2103.14443/) (2021)
- [KGE-CL: Contrastive Learning of Tensor Decomposition Based Knowledge Graph Embeddings](https://mlsanity.com/paper/2112.04871/) (2021)
- [Lexical semantics enhanced neural word embeddings](https://mlsanity.com/paper/2210.00754/) (2022)
- [3D Rotation and Translation for Hyperbolic Knowledge Graph Embedding](https://mlsanity.com/paper/2305.13015/) (2023)
- [Navigating Healthcare Insights: A Birds Eye View of Explainability with Knowledge Graphs](https://mlsanity.com/paper/2309.16593/) (2023)
- [Block-Diagonal Orthogonal Relation and Matrix Entity for Knowledge Graph Embedding](https://mlsanity.com/paper/2401.05967/) (2024)
- [Optimal Embedding Guided Negative Sample Generation for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2504.03327/) (2025)
- [CorporateBench: Large-Scale Q&A Benchmarking with Temporal Knowledge Bases](https://mlsanity.com/paper/2608.27391/) (2026)

## Abstract (arXiv)

This paper focuses on how to take advantage of external knowledge bases (KBs) to improve recurrent neural networks for machine reading. Traditional methods that exploit knowledge from KBs encode knowledge as discrete indicator features. Not only do these features generalize poorly, but they require task-specific feature engineering to achieve good performance. We propose KBLSTM, a novel neural model that leverages continuous representations of KBs to enhance the learning of recurrent neural networks for machine reading. To effectively integrate background knowledge with information from the currently processed text, our model employs an attention mechanism with a sentinel to adaptively decide whether to attend to background knowledge and which information from KBs is useful. Experimental results show that our model achieves accuracies that surpass the previous state-of-the-art results for both entity extraction and event extraction on the widely used ACE2005 dataset.