# Clustering Text Using Attention

arXiv: https://arxiv.org/abs/2201.02816 · HTML: https://mlsanity.com/paper/2201.02816/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 8 Jan 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [A Survey of Heterogeneous Information Network Analysis](https://mlsanity.com/paper/1511.04854/) (2015)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems](https://mlsanity.com/paper/1512.08756/) (2015)
- [Vector representations of text data in deep learning](https://mlsanity.com/paper/1901.01695/) (2019)
- [Mutual Clustering on Comparative Texts via Heterogeneous Information Networks](https://mlsanity.com/paper/1903.03762/) (2019)

## Abstract (arXiv)

Clustering Text has been an important problem in the domain of Natural Language Processing. While there are techniques to cluster text based on using conventional clustering techniques on top of contextual or non-contextual vector space representations, it still remains a prevalent area of research possible to various improvements in performance and implementation of these techniques. This paper discusses a novel technique to cluster text using attention mechanisms. Attention Mechanisms have proven to be highly effective in various NLP tasks in recent times. This paper extends the idea of attention mechanism in clustering space and sheds some light on a whole new area of research