# Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition

arXiv: https://arxiv.org/abs/2204.00465 · HTML: https://mlsanity.com/paper/2204.00465/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends](https://mlsanity.com/paper/2001.00378/) (2020)
- [Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion](https://mlsanity.com/paper/2203.16705/) (2022)
- [A Neural Network Alternative to Non-Negative Audio Models](https://mlsanity.com/paper/1609.03296/) (2016)
- [Masked Proxy Loss For Text-Independent Speaker Verification](https://mlsanity.com/paper/2011.04491/) (2020)

## Built on by (most cited first, 12 of 12)

- [AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations](https://mlsanity.com/paper/2302.06419/) (2023)
- [Coding Speech through Vocal Tract Kinematics](https://mlsanity.com/paper/2406.12998/) (2024)
- [Self-Supervised Models of Speech Infer Universal Articulatory Kinematics](https://mlsanity.com/paper/2310.10788/) (2023)
- [YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection](https://mlsanity.com/paper/2408.15297/) (2024)
- [Towards Hierarchical Spoken Language Dysfluency Modeling](https://mlsanity.com/paper/2401.10015/) (2024)
- [SSDM: Scalable Speech Dysfluency Modeling](https://mlsanity.com/paper/2408.16221/) (2024)
- [Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection](https://mlsanity.com/paper/2409.09621/) (2024)
- [Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection](https://mlsanity.com/paper/2505.22029/) (2025)
- [Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection](https://mlsanity.com/paper/2409.13582/) (2024)
- [Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detection](https://mlsanity.com/paper/2505.16351/) (2025)
- [Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis](https://mlsanity.com/paper/2506.12073/) (2025)
- [K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function](https://mlsanity.com/paper/2507.03043/) (2025)

## Abstract (arXiv)

Most of the research on data-driven speech representation learning has focused on raw audios in an end-to-end manner, paying little attention to their internal phonological or gestural structure. This work, investigating the speech representations derived from articulatory kinematics signals, uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. By applying sparse constraints, the gestural scores leverage the discrete combinatorial properties of phonological gestures. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully. The proposed work thus makes a bridge between articulatory phonology and deep neural networks to leverage informative, intelligible, interpretable,and efficient speech representations.