# Siamese CBOW: Optimizing Word Embeddings for Sentence Representations

arXiv: https://arxiv.org/abs/1606.04640 · HTML: https://mlsanity.com/paper/1606.04640/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2016.

## Built on by (most cited first, 10 of 10)

- [Semi-automatic Generation of Multilingual Datasets for Stance Detection in Twitter](https://mlsanity.com/paper/2101.11978/) (2021)
- [Efficacy of BERT embeddings on predicting disaster from Twitter data](https://mlsanity.com/paper/2108.10698/) (2021)
- [Cross Modal Data Discovery over Structured and Unstructured Data Lakes](https://mlsanity.com/paper/2306.00932/) (2023)
- [Language Independent Stance Detection: Social Interaction-based Embeddings and Large Language Models](https://mlsanity.com/paper/2210.05715/) (2022)
- [Hyperbolic sentence representations for solving Textual Entailment](https://mlsanity.com/paper/2406.15472/) (2024)
- [Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning](https://mlsanity.com/paper/2311.17514/) (2023)
- [Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations](https://mlsanity.com/paper/2106.00851/) (2021)
- [Measuring a Texts Fairness Dimensions Using Machine Learning Based on Social Psychological Factors](https://mlsanity.com/paper/2111.00086/) (2021)
- [Learning Analytics from Spoken Discussion Dialogs in Flipped Classroom](https://mlsanity.com/paper/2301.12399/) (2023)
- [The Illusion of Friendship: Why Generative AI Demands Unprecedented Ethical Vigilance](https://mlsanity.com/paper/2601.08874/) (2026)

## Abstract (arXiv)

We present the Siamese Continuous Bag of Words (Siamese CBOW) model, a neural network for efficient estimation of high-quality sentence embeddings. Averaging the embeddings of words in a sentence has proven to be a surprisingly successful and efficient way of obtaining sentence embeddings. However, word embeddings trained with the methods currently available are not optimized for the task of sentence representation, and, thus, likely to be suboptimal. Siamese CBOW handles this problem by training word embeddings directly for the purpose of being averaged. The underlying neural network learns word embeddings by predicting, from a sentence representation, its surrounding sentences. We show the robustness of the Siamese CBOW model by evaluating it on 20 datasets stemming from a wide variety of sources.