# Double Articulation Analyzer with Prosody for Unsupervised Word and Phoneme Discovery

arXiv: https://arxiv.org/abs/2103.08199 · HTML: https://mlsanity.com/paper/2103.08199/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 76 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 23 of 23)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge](https://mlsanity.com/paper/2005.09409/) (2020)
- [The Zero Resource Speech Challenge 2017](https://mlsanity.com/paper/1712.04313/) (2017)
- [Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech](https://mlsanity.com/paper/1911.09602/) (2019)
- [Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation](https://mlsanity.com/paper/2007.13465/) (2020)
- [The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units](https://mlsanity.com/paper/2010.05967/) (2020)
- [Representations of language in a model of visually grounded speech signal](https://mlsanity.com/paper/1702.01991/) (2017)
- [Symbol Emergence in Robotics: A Survey](https://mlsanity.com/paper/1509.08973/) (2015)
- [Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models](https://mlsanity.com/paper/1811.00403/) (2018)
- [A segmental framework for fully-unsupervised large-vocabulary speech recognition](https://mlsanity.com/paper/1606.06950/) (2016)
- [Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks](https://mlsanity.com/paper/2012.07551/) (2020)
- [Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metrics](https://mlsanity.com/paper/2008.00731/) (2020)
- [Improved and Scalable Online Learning of Spatial Concepts and Language Models with Mapping](https://mlsanity.com/paper/1803.03481/) (2018)
- [Predictive Processing in Cognitive Robotics: a Review](https://mlsanity.com/paper/2101.06611/) (2021)
- [Semantic query-by-example speech search using visual grounding](https://mlsanity.com/paper/1904.07078/) (2019)
- [Online Spatial Concept and Lexical Acquisition with Simultaneous Localization and Mapping](https://mlsanity.com/paper/1704.04664/) (2017)
- [On the Use/Misuse of the Term 'Phoneme'](https://mlsanity.com/paper/1907.11640/) (2019)
- [A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery](https://mlsanity.com/paper/2011.03115/) (2020)
- [Towards localisation of keywords in speech using weak supervision](https://mlsanity.com/paper/2012.07396/) (2020)
- [Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition from Continuous Speech Signals](https://mlsanity.com/paper/1506.06646/) (2015)
- [Spatial Concept Acquisition for a Mobile Robot that Integrates Self-Localization and Unsupervised Word Discovery from Spoken Sentences](https://mlsanity.com/paper/1602.01208/) (2016)
- [Hippocampal formation-inspired probabilistic generative model](https://mlsanity.com/paper/2103.07356/) (2021)
- [Perception of prosodic variation for speech synthesis using an unsupervised discrete representation of F0](https://mlsanity.com/paper/2003.06686/) (2020)

## Built on by (most cited first, 7 of 7)

- [Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring](https://mlsanity.com/paper/2202.11929/) (2022)
- [Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programming](https://mlsanity.com/paper/2409.14486/) (2024)
- [Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?](https://mlsanity.com/paper/2507.19204/) (2025)
- [Unsupervised lexicon learning from speech is limited by representations rather than clustering](https://mlsanity.com/paper/2510.09225/) (2025)
- [Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues](https://mlsanity.com/paper/2201.06786/) (2022)
- [Brain-inspired probabilistic generative model for double articulation analysis of spoken language](https://mlsanity.com/paper/2207.02457/) (2022)
- [Recovering the Zipfian Distribution in Unsupervised Term Discovery](https://mlsanity.com/paper/2606.10781/) (2026)

## Abstract (arXiv)

Infants acquire words and phonemes from unsegmented speech signals using segmentation cues, such as distributional, prosodic, and co-occurrence cues. Many pre-existing computational models that represent the process tend to focus on distributional or prosodic cues. This paper proposes a nonparametric Bayesian probabilistic generative model called the prosodic hierarchical Dirichlet process-hidden language model (Prosodic HDP-HLM). Prosodic HDP-HLM, an extension of HDP-HLM, considers both prosodic and distributional cues within a single integrative generative model. We conducted three experiments on different types of datasets, and demonstrate the validity of the proposed method. The results show that the Prosodic DAA successfully uses prosodic cues and outperforms a method that solely uses distributional cues. The main contributions of this study are as follows: 1) We develop a probabilistic generative model for time series data including prosody that potentially has a double articulation structure; 2) We propose the Prosodic DAA by deriving the inference procedure for Prosodic HDP-HLM and show that Prosodic DAA can discover words directly from continuous human speech signals using statistical information and prosodic information in an unsupervised manner; 3) We show that prosodic cues contribute to word segmentation more in naturally distributed case words, i.e., they follow Zipf's law.