# Contextual Word Representations: A Contextual Introduction

arXiv: https://arxiv.org/abs/1902.06006 · HTML: https://mlsanity.com/paper/1902.06006/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2019.

## Built on by (most cited first, 6 of 6)

- [CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation](https://mlsanity.com/paper/2406.14830/) (2024)
- [Neural Language Models for Nineteenth-Century English](https://mlsanity.com/paper/2105.11321/) (2021)
- [Language Models as Zero-shot Visual Semantic Learners](https://mlsanity.com/paper/2107.12021/) (2021)
- [Word Embeddings for Banking Industry](https://mlsanity.com/paper/2306.01807/) (2023)
- [A survey of neural-network-based methods utilising comparable data for finding translation equivalents](https://mlsanity.com/paper/2410.15144/) (2024)
- [Gyan: An Explainable Neuro-Symbolic Language Model](https://mlsanity.com/paper/2605.04759/) (2026)

## Abstract (arXiv)

This introduction aims to tell the story of how we put words into computers. It is part of the story of the field of natural language processing (NLP), a branch of artificial intelligence. It targets a wide audience with a basic understanding of computer programming, but avoids a detailed mathematical treatment, and it does not present any algorithms. It also does not focus on any particular application of NLP such as translation, question answering, or information extraction. The ideas presented here were developed by many researchers over many decades, so the citations are not exhaustive but rather direct the reader to a handful of papers that are, in the author's view, seminal. After reading this document, you should have a general understanding of word vectors (also known as word embeddings): why they exist, what problems they solve, where they come from, how they have changed over time, and what some of the open questions about them are. Readers already familiar with word vectors are advised to skip to Section 5 for the discussion of the most recent advance, contextual word vectors.