# Ultradense Word Embeddings by Orthogonal Transformation

arXiv: https://arxiv.org/abs/1602.07572 · HTML: https://mlsanity.com/paper/1602.07572/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2016.

## Built on by (most cited first, 8 of 8)

- [Locating Language-Specific Information in Contextualized Embeddings](https://mlsanity.com/paper/2109.08040/) (2021)
- [Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces](https://mlsanity.com/paper/2106.07505/) (2021)
- [Guilt by Association: Emotion Intensities in Lexical Representations](https://mlsanity.com/paper/2104.08679/) (2021)
- [Explainability of Text Processing and Retrieval Methods: A Survey](https://mlsanity.com/paper/2212.07126/) (2022)
- [Dialectograms: Machine Learning Differences between Discursive Communities](https://mlsanity.com/paper/2302.05657/) (2023)
- [Bridging Natural Language Processing and Psycholinguistics: computationally grounded semantic similarity datasets for Basque and Spanish](https://mlsanity.com/paper/2304.09616/) (2023)
- [Emotion Embeddings unicodex2014 Learning Stable and Homogeneous Abstractions from Heterogeneous Affective Datasets](https://mlsanity.com/paper/2308.07871/) (2023)
- [Optimal synthesis embeddings](https://mlsanity.com/paper/2406.10259/) (2024)

## Abstract (arXiv)

Embeddings are generic representations that are useful for many NLP tasks. In this paper, we introduce DENSIFIER, a method that learns an orthogonal transformation of the embedding space that focuses the information relevant for a task in an ultradense subspace of a dimensionality that is smaller by a factor of 100 than the original space. We show that ultradense embeddings generated by DENSIFIER reach state of the art on a lexicon creation task in which words are annotated with three types of lexical information - sentiment, concreteness and frequency. On the SemEval2015 10B sentiment analysis task we show that no information is lost when the ultradense subspace is used, but training is an order of magnitude more efficient due to the compactness of the ultradense space.