# Building Language Models for Text with Named Entities

arXiv: https://arxiv.org/abs/1805.04836 · HTML: https://mlsanity.com/paper/1805.04836/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2018.

## Built on by (most cited first, 15 of 15)

- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [MapCoder: Multi-Agent Code Generation for Competitive Problem Solving](https://mlsanity.com/paper/2405.11403/) (2024)
- [Retrieval Augmented Code Generation and Summarization](https://mlsanity.com/paper/2108.11601/) (2021)
- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://mlsanity.com/paper/2303.03004/) (2023)
- [CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging](https://mlsanity.com/paper/2502.05664/) (2025)
- [On Multi-Modal Learning of Editing Source Code](https://mlsanity.com/paper/2108.06645/) (2021)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [Ratatouille: A tool for Novel Recipe Generation](https://mlsanity.com/paper/2206.08267/) (2022)
- [SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors](https://mlsanity.com/paper/2502.11167/) (2025)
- [Towards Continual Entity Learning in Language Models for Conversational Agents](https://mlsanity.com/paper/2108.00082/) (2021)
- [Towards Consistent Language Models Using Declarative Constraints](https://mlsanity.com/paper/2312.15472/) (2023)
- [Poison with Style: A Practical Poisoning Attack on Code Large Language Models](https://mlsanity.com/paper/2605.27631/) (2026)
- [The Digital Sous Chef -- A Comparative Study on Fine-Tuning Language Models for Recipe Generation](https://mlsanity.com/paper/2508.14718/) (2025)
- [Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics](https://mlsanity.com/paper/2604.17715/) (2026)

## Abstract (arXiv)

Text in many domains involves a significant amount of named entities. Predict- ing the entity names is often challenging for a language model as they appear less frequent on the training corpus. In this paper, we propose a novel and effective approach to building a discriminative language model which can learn the entity names by leveraging their entity type information. We also introduce two benchmark datasets based on recipes and Java programming codes, on which we evalu- ate the proposed model. Experimental re- sults show that our model achieves 52.2% better perplexity in recipe generation and 22.06% on code generation than the state-of-the-art language models.