# Representation learning for very short texts using weighted word embedding aggregation

arXiv: https://arxiv.org/abs/1607.00570 · HTML: https://mlsanity.com/paper/1607.00570/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2016.

## Built on by (most cited first, 12 of 12)

- [Unsupervised Matching of Data and Text](https://mlsanity.com/paper/2112.08776/) (2021)
- [The Use of NLP-Based Text Representation Techniques to Support Requirement Engineering Tasks: A Systematic Mapping Review](https://mlsanity.com/paper/2206.00421/) (2022)
- [Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applications](https://mlsanity.com/paper/2306.16143/) (2023)
- [Task-Specific Dependency-based Word Embedding Methods](https://mlsanity.com/paper/2110.13376/) (2021)
- [Generalized Funnelling: Ensemble Learning and Heterogeneous Document Embeddings for Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.14764/) (2021)
- [From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space](https://mlsanity.com/paper/2409.10528/) (2024)
- [Merchandise Recommendation for Retail Events with Word Embedding Weighted Tf-idf and Dynamic Query Expansion](https://mlsanity.com/paper/2208.08581/) (2022)
- [Uzbek text's correspondence with the educational potential of pupils: a case study of the School corpus](https://mlsanity.com/paper/2303.00465/) (2023)
- [CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering](https://mlsanity.com/paper/2304.11061/) (2023)
- [Explainable Graph Spectral Clustering For GloVe-like Text Embeddings](https://mlsanity.com/paper/2508.14075/) (2025)
- [Who is a Better Matchmaker? Human vs. Algorithmic Judge Assignment in a High-Stakes Startup Competition](https://mlsanity.com/paper/2510.12692/) (2025)
- [An Efficient Classification Model for Cyber Text](https://mlsanity.com/paper/2511.03107/) (2025)

## Abstract (arXiv)

Short text messages such as tweets are very noisy and sparse in their use of vocabulary. Traditional textual representations, such as tf-idf, have difficulty grasping the semantic meaning of such texts, which is important in applications such as event detection, opinion mining, news recommendation, etc. We constructed a method based on semantic word embeddings and frequency information to arrive at low-dimensional representations for short texts designed to capture semantic similarity. For this purpose we designed a weight-based model and a learning procedure based on a novel median-based loss function. This paper discusses the details of our model and the optimization methods, together with the experimental results on both Wikipedia and Twitter data. We find that our method outperforms the baseline approaches in the experiments, and that it generalizes well on different word embeddings without retraining. Our method is therefore capable of retaining most of the semantic information in the text, and is applicable out-of-the-box.