# Language Models with Pre-Trained (GloVe) Word Embeddings

arXiv: https://arxiv.org/abs/1610.03759 · HTML: https://mlsanity.com/paper/1610.03759/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2016.

## Built on by (most cited first, 2 of 2)

- [From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models](https://mlsanity.com/paper/2411.05036/) (2024)
- [Combining Metric Learning and Attention Heads For Accurate and Efficient Multilabel Image Classification](https://mlsanity.com/paper/2209.06585/) (2022)

## Abstract (arXiv)

In this work we implement a training of a Language Model (LM), using Recurrent Neural Network (RNN) and GloVe word embeddings, introduced by Pennigton et al. in [1]. The implementation is following the general idea of training RNNs for LM tasks presented in [2], but is rather using Gated Recurrent Unit (GRU) [3] for a memory cell, and not the more commonly used LSTM [4].