# A Survey on Neural Network Language Models

arXiv: https://arxiv.org/abs/1906.03591 · HTML: https://mlsanity.com/paper/1906.03591/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2019.

## Built on by (most cited first, 12 of 12)

- [Algorithmic progress in language models](https://mlsanity.com/paper/2403.05812/) (2024)
- [An enriched category theory of language: from syntax to semantics](https://mlsanity.com/paper/2106.07890/) (2021)
- [Customizing Contextualized Language Models forLegal Document Reviews](https://mlsanity.com/paper/2102.05757/) (2021)
- [Ticket-BERT: Labeling Incident Management Tickets with Language Models](https://mlsanity.com/paper/2307.00108/) (2023)
- [An Introduction to Natural Language Processing Techniques and Framework for Clinical Implementation in Radiation Oncology](https://mlsanity.com/paper/2311.02205/) (2023)
- [Res-CNN-BiLSTM Network for overcoming Mental Health Disturbances caused due to Cyberbullying through Social Media](https://mlsanity.com/paper/2204.09738/) (2022)
- [A Pilot Evaluation of ChatGPT and DALL-E 2 on Decision Making and Spatial Reasoning](https://mlsanity.com/paper/2302.09068/) (2023)
- [When Giant Language Brains Just Aren't Enough! Domain Pizzazz with Knowledge Sparkle Dust](https://mlsanity.com/paper/2305.07230/) (2023)
- [Feature Alignment as a Generative Process](https://mlsanity.com/paper/2106.12562/) (2021)
- [Sentence-level Online Handwritten Chinese Character Recognition](https://mlsanity.com/paper/2108.02561/) (2021)
- [Structured Thoughts Automaton: First Formalized Execution Model for Auto-Regressive Language Models](https://mlsanity.com/paper/2306.10196/) (2023)
- [An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks](https://mlsanity.com/paper/2401.00582/) (2023)

## Abstract (arXiv)

As the core component of Natural Language Processing (NLP) system, Language Model (LM) can provide word representation and probability indication of word sequences. Neural Network Language Models (NNLMs) overcome the curse of dimensionality and improve the performance of traditional LMs. A survey on NNLMs is performed in this paper. The structure of classic NNLMs is described firstly, and then some major improvements are introduced and analyzed. We summarize and compare corpora and toolkits of NNLMs. Further, some research directions of NNLMs are discussed.