# CommunityFish: A Poisson-based Document Scaling With Hierarchical Clustering

arXiv: https://arxiv.org/abs/2308.14873 · HTML: https://mlsanity.com/paper/2308.14873/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 6 of 6)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Text-Based Ideal Points](https://mlsanity.com/paper/2005.04232/) (2020)
- [Keyword Assisted Topic Models](https://mlsanity.com/paper/2004.05964/) (2020)
- [Political Text Scaling Meets Computational Semantics](https://mlsanity.com/paper/1904.06217/) (2019)
- [Topic Scaling: A Joint Document Scaling -- Topic Model Approach To Learn Time-Specific Topics](https://mlsanity.com/paper/2104.01117/) (2021)

## Abstract (arXiv)

Document scaling has been a key component in text-as-data applications for social scientists and a major field of interest for political researchers, who aim at uncovering differences between speakers or parties with the help of different probabilistic and non-probabilistic approaches. Yet, most of these techniques are either built upon the agnostically bag-of-word hypothesis or use prior information borrowed from external sources that might embed the results with a significant bias. If the corpus has long been considered as a collection of documents, it can also be seen as a dense network of connected words whose structure could be clustered to differentiate independent groups of words, based on their co-occurrences in documents, known as communities. This paper introduces CommunityFish as an augmented version of Wordfish based on a hierarchical clustering, namely the Louvain algorithm, on the word space to yield communities as semantic and independent n-grams emerging from the corpus and use them as an input to Wordfish method, instead of considering the word space. This strategy emphasizes the interpretability of the results, since communities have a non-overlapping structure, hence a crucial informative power in discriminating parties or speakers, in addition to allowing a faster execution of the Poisson scaling model. Aside from yielding communities, assumed to be subtopic proxies, the application of this technique outperforms the classic Wordfish model by highlighting historical developments in the U.S. State of the Union addresses and was found to replicate the prevailing political stance in Germany when using the corpus of parties' legislative manifestos.