# KATE: K-Competitive Autoencoder for Text

arXiv: https://arxiv.org/abs/1705.02033 · HTML: https://mlsanity.com/paper/1705.02033/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2017.

## Built on by (most cited first, 5 of 5)

- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [DINE: Dimensional Interpretability of Node Embeddings](https://mlsanity.com/paper/2310.01162/) (2023)
- [Disentangled and Self-Explainable Node Representation Learning](https://mlsanity.com/paper/2410.21043/) (2024)
- [Multi-environment Topic Models](https://mlsanity.com/paper/2410.24126/) (2024)
- [A General Framework for Learning Prosodic-Enhanced Representation of Rap Lyrics](https://mlsanity.com/paper/2103.12615/) (2021)

## Abstract (arXiv)

Autoencoders have been successful in learning meaningful representations from image datasets. However, their performance on text datasets has not been widely studied. Traditional autoencoders tend to learn possibly trivial representations of text documents due to their confounding properties such as high-dimensionality, sparsity and power-law word distributions. In this paper, we propose a novel k-competitive autoencoder, called KATE, for text documents. Due to the competition between the neurons in the hidden layer, each neuron becomes specialized in recognizing specific data patterns, and overall the model can learn meaningful representations of textual data. A comprehensive set of experiments show that KATE can learn better representations than traditional autoencoders including denoising, contractive, variational, and k-sparse autoencoders. Our model also outperforms deep generative models, probabilistic topic models, and even word representation models (e.g., Word2Vec) in terms of several downstream tasks such as document classification, regression, and retrieval.