# Incremental Skip-gram Model with Negative Sampling

arXiv: https://arxiv.org/abs/1704.03956 · HTML: https://mlsanity.com/paper/1704.03956/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2017.

## Built on by (most cited first, 4 of 4)

- [Continual learning in cross-modal retrieval](https://mlsanity.com/paper/2104.06806/) (2021)
- [Short term prediction of demand for ride hailing services: A deep learning approach](https://mlsanity.com/paper/2212.03956/) (2022)
- [RiverText: A Python Library for Training and Evaluating Incremental Word Embeddings from Text Data Streams](https://mlsanity.com/paper/2506.23192/) (2025)
- [AETAS: Analysis of Evolving Temporal Affect and Semantics for Legal History](https://mlsanity.com/paper/2512.22196/) (2025)

## Abstract (arXiv)

This paper explores an incremental training strategy for the skip-gram model with negative sampling (SGNS) from both empirical and theoretical perspectives. Existing methods of neural word embeddings, including SGNS, are multi-pass algorithms and thus cannot perform incremental model update. To address this problem, we present a simple incremental extension of SGNS and provide a thorough theoretical analysis to demonstrate its validity. Empirical experiments demonstrated the correctness of the theoretical analysis as well as the practical usefulness of the incremental algorithm.