# Latent Relation Language Models

arXiv: https://arxiv.org/abs/1908.07690 · HTML: https://mlsanity.com/paper/1908.07690/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling](https://mlsanity.com/paper/2306.11489/) (2023)
- [Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey](https://mlsanity.com/paper/2110.08455/) (2021)
- [SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining](https://mlsanity.com/paper/2108.08983/) (2021)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge](https://mlsanity.com/paper/2109.12573/) (2021)
- [Capturing Structural Locality in Non-parametric Language Models](https://mlsanity.com/paper/2110.02870/) (2021)
- [A Discriminative Entity-Aware Language Model for Virtual Assistants](https://mlsanity.com/paper/2106.11292/) (2021)
- [Space-Efficient Representation of Entity-centric Query Language Models](https://mlsanity.com/paper/2206.14885/) (2022)
- [Modeling Spoken Information Queries for Virtual Assistants: Open Problems, Challenges and Opportunities](https://mlsanity.com/paper/2304.13149/) (2023)
- [Language Model Evaluation in Open-ended Text Generation](https://mlsanity.com/paper/2108.03578/) (2021)
- [Open-domain Dialogue Generation Grounded with Dynamic Multi-form Knowledge Fusion](https://mlsanity.com/paper/2204.11239/) (2022)
- [RelBERT: Embedding Relations with Language Models](https://mlsanity.com/paper/2310.00299/) (2023)
- [Knowledge Graph Completion with Text-aided Regularization](https://mlsanity.com/paper/2101.08962/) (2021)
- [Empirical analysis of binding precedent efficiency in Brazilian Supreme Court via case classification](https://mlsanity.com/paper/2407.07004/) (2024)

## Abstract (arXiv)

In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations. This model has a number of attractive properties: it not only improves language modeling performance, but is also able to annotate the posterior probability of entity spans for a given text through relations. Experiments demonstrate empirical improvements over both a word-based baseline language model and a previous approach that incorporates knowledge graph information. Qualitative analysis further demonstrates the proposed model's ability to learn to predict appropriate relations in context.