# Recovering the Zipfian Distribution in Unsupervised Term Discovery

arXiv: https://arxiv.org/abs/2606.10781 · HTML: https://mlsanity.com/paper/2606.10781/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 27 of 27)

- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [The Zero Resource Speech Challenge 2017](https://mlsanity.com/paper/1712.04313/) (2017)
- [The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units](https://mlsanity.com/paper/2010.05967/) (2020)
- [Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation](https://mlsanity.com/paper/2106.02170/) (2021)
- [An embedded segmental K-means model for unsupervised segmentation and clustering of speech](https://mlsanity.com/paper/1703.08135/) (2017)
- [Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring](https://mlsanity.com/paper/2202.11929/) (2022)
- [SyllableLM: Learning Coarse Semantic Units for Speech Language Models](https://mlsanity.com/paper/2410.04029/) (2024)
- [Sylber: Syllabic Embedding Representation of Speech from Raw Audio](https://mlsanity.com/paper/2410.07168/) (2024)
- [Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words](https://mlsanity.com/paper/2110.15909/) (2021)
- [SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT](https://mlsanity.com/paper/2310.10803/) (2023)
- [Self-Expressing Autoencoders for Unsupervised Spoken Term Discovery](https://mlsanity.com/paper/2007.13033/) (2020)
- [DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon](https://mlsanity.com/paper/2206.11332/) (2022)
- [Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model](https://mlsanity.com/paper/2305.11435/) (2023)
- [Spoken-Term Discovery using Discrete Speech Units](https://mlsanity.com/paper/2408.14390/) (2024)
- [Double Articulation Analyzer with Prosody for Unsupervised Word and Phoneme Discovery](https://mlsanity.com/paper/2103.08199/) (2021)
- [Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programming](https://mlsanity.com/paper/2409.14486/) (2024)
- [ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modeling](https://mlsanity.com/paper/2602.15537/) (2026)
- [Unsupervised lexicon learning from speech is limited by representations rather than clustering](https://mlsanity.com/paper/2510.09225/) (2025)
- [Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings](https://mlsanity.com/paper/2307.02083/) (2023)
- [Low-resource keyword spotting using contrastively trained transformer acoustic word embeddings](https://mlsanity.com/paper/2506.17690/) (2025)
- [Revisiting Lexicon Evaluation in Unsupervised Word Discovery](https://mlsanity.com/paper/2606.06183/) (2026)

## Abstract (arXiv)

Unsupervised term discovery involves segmenting unlabelled speech into word- or syllable-like units and clustering these into a lexicon of candidate types. True lexicons follow a Zipfian distribution, yet the dominant centre-based clustering approach -- K-means -- produces a more uniform distribution due to an inductive bias toward spherical clusters. In this paper we revisit graph-based clustering as a bottom-up alternative, where segment embeddings are connected by pairwise similarity and partitioned using the Leiden algorithm. We show that graph clustering substantially outperforms centre-based approaches (K-means, GMM, BIRCH) in both word- and syllable-level lexicon discovery across three languages, producing more Zipf-like distributions. Another bottom-up approach, agglomerative clustering with average linkage, also performs well, although it is computationally less efficient and allows for less control over the resulting distribution. Our work calls into question the dominance of centre-based clustering for term discovery, and promotes graph clustering as an attractive alternative.