# Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition

arXiv: https://arxiv.org/abs/1904.04458 · HTML: https://mlsanity.com/paper/1904.04458/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [K-LITE: Learning Transferable Visual Models with External Knowledge](https://mlsanity.com/paper/2204.09222/) (2022)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [Doubly Right Object Recognition: A Why Prompt for Visual Rationales](https://mlsanity.com/paper/2212.06202/) (2022)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Can Language Representation Models Think in Bets?](https://mlsanity.com/paper/2210.07519/) (2022)
- [A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2101.00388/) (2021)
- [Towards Continual Entity Learning in Language Models for Conversational Agents](https://mlsanity.com/paper/2108.00082/) (2021)
- [Few-shot Named Entity Recognition with Cloze Questions](https://mlsanity.com/paper/2111.12421/) (2021)
- [Knowledge Pyramid Construction for Multi-Level Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.21276/) (2024)
- [CoreLM: Coreference-aware Language Model Fine-Tuning](https://mlsanity.com/paper/2111.02687/) (2021)

## Abstract (arXiv)

Traditional language models are unable to efficiently model entity names observed in text. All but the most popular named entities appear infrequently in text providing insufficient context. Recent efforts have recognized that context can be generalized between entity names that share the same type (e.g., person or location) and have equipped language models with access to an external knowledge base (KB). Our Knowledge-Augmented Language Model (KALM) continues this line of work by augmenting a traditional model with a KB. Unlike previous methods, however, we train with an end-to-end predictive objective optimizing the perplexity of text. We do not require any additional information such as named entity tags. In addition to improving language modeling performance, KALM learns to recognize named entities in an entirely unsupervised way by using entity type information latent in the model. On a Named Entity Recognition (NER) task, KALM achieves performance comparable with state-of-the-art supervised models. Our work demonstrates that named entities (and possibly other types of world knowledge) can be modeled successfully using predictive learning and training on large corpora of text without any additional information.