# A Primer on Neural Network Models for Natural Language Processing

arXiv: https://arxiv.org/abs/1510.00726 · HTML: https://mlsanity.com/paper/1510.00726/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2015.

## Built on by (most cited first, 30 of 76)

- [Large Language Models in Mental Health Care: a Scoping Review](https://mlsanity.com/paper/2401.02984/) (2024)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [DeepHate: Hate Speech Detection via Multi-Faceted Text Representations](https://mlsanity.com/paper/2103.11799/) (2021)
- [Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?](https://mlsanity.com/paper/2310.11616/) (2023)
- [Watermarking Graph Neural Networks based on Backdoor Attacks](https://mlsanity.com/paper/2110.11024/) (2021)
- [Quantum Recurrent Neural Networks for Sequential Learning](https://mlsanity.com/paper/2302.03244/) (2023)
- [Conformal prediction for text infilling and part-of-speech prediction](https://mlsanity.com/paper/2111.02592/) (2021)
- [The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?](https://mlsanity.com/paper/2305.07970/) (2023)
- [A survey on text generation using generative adversarial networks](https://mlsanity.com/paper/2212.11119/) (2022)
- [Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streams](https://mlsanity.com/paper/2304.10740/) (2023)
- [Exploring Multimodal Approaches for Alzheimer's Disease Detection Using Patient Speech Transcript and Audio Data](https://mlsanity.com/paper/2307.02514/) (2023)
- [A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries](https://mlsanity.com/paper/2107.10652/) (2021)
- [Language Models as a Service: Overview of a New Paradigm and its Challenges](https://mlsanity.com/paper/2309.16573/) (2023)
- [Exploring Transformer Based Models to Identify Hate Speech and Offensive Content in English and Indo-Aryan Languages](https://mlsanity.com/paper/2111.13974/) (2021)
- [A Survey of Large Language Models in Cybersecurity](https://mlsanity.com/paper/2402.16968/) (2024)
- [Realised Volatility Forecasting: Machine Learning via Financial Word Embedding](https://mlsanity.com/paper/2108.00480/) (2021)
- [Rationally Inattentive Inverse Reinforcement Learning Explains YouTube Commenting Behavior](https://mlsanity.com/paper/1910.11703/) (2019)
- [Introduction to Neural Transfer Learning with Transformers for Social Science Text Analysis](https://mlsanity.com/paper/2102.02111/) (2021)
- [Abusive and Threatening Language Detection in Urdu using Boosting based and BERT based models: A Comparative Approach](https://mlsanity.com/paper/2111.14830/) (2021)
- [Trace Encoding in Process Mining: a survey and benchmarking](https://mlsanity.com/paper/2301.02167/) (2023)
- [Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics](https://mlsanity.com/paper/2404.19178/) (2024)
- [Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understanding](https://mlsanity.com/paper/2507.08031/) (2025)
- [Stance Detection with BERT Embeddings for Credibility Analysis of Information on Social Media](https://mlsanity.com/paper/2105.10272/) (2021)
- [A scalable deep learning approach for solving high-dimensional dynamic optimal transport](https://mlsanity.com/paper/2205.07521/) (2022)
- [A Statistical-Modelling Approach to Feedforward Neural Network Model Selection](https://mlsanity.com/paper/2207.04248/) (2022)
- [Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey](https://mlsanity.com/paper/2105.06300/) (2021)
- [Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directions](https://mlsanity.com/paper/2302.06801/) (2023)
- [Exploring Classic and Neural Lexical Translation Models for Information Retrieval: Interpretability, Effectiveness, and Efficiency Benefits](https://mlsanity.com/paper/2102.06815/) (2021)
- [Semi-tensor Product-based TensorDecomposition for Neural Network Compression](https://mlsanity.com/paper/2109.15200/) (2021)
- [Deep Learning Architecture for Automatic Essay Scoring](https://mlsanity.com/paper/2206.08232/) (2022)

## Abstract (arXiv)

Over the past few years, neural networks have re-emerged as powerful machine-learning models, yielding state-of-the-art results in fields such as image recognition and speech processing. More recently, neural network models started to be applied also to textual natural language signals, again with very promising results. This tutorial surveys neural network models from the perspective of natural language processing research, in an attempt to bring natural-language researchers up to speed with the neural techniques. The tutorial covers input encoding for natural language tasks, feed-forward networks, convolutional networks, recurrent networks and recursive networks, as well as the computation graph abstraction for automatic gradient computation.