# Untangling in Invariant Speech Recognition

arXiv: https://arxiv.org/abs/2003.01787 · HTML: https://mlsanity.com/paper/2003.01787/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 13 of 13)

- [On the geometry of generalization and memorization in deep neural networks](https://mlsanity.com/paper/2105.14602/) (2021)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2303.03307/) (2023)
- [The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models](https://mlsanity.com/paper/2502.08009/) (2025)
- [Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations](https://mlsanity.com/paper/2406.09200/) (2024)
- [Neural Population Geometry Reveals the Role of Stochasticity in Robust Perception](https://mlsanity.com/paper/2111.06979/) (2021)
- [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025)
- [Learning Defense Transformers for Counterattacking Adversarial Examples](https://mlsanity.com/paper/2103.07595/) (2021)
- [Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks](https://mlsanity.com/paper/2108.12001/) (2021)
- [Emergent Manifold Separability during Reasoning in Large Language Models](https://mlsanity.com/paper/2602.20338/) (2026)
- [Automated Mathematical Equation Structure Discovery for Visual Analysis](https://mlsanity.com/paper/2104.08633/) (2021)
- [Deep Speech Representations Track Word Timing along a Low-Dimensional Circular Trajectory](https://mlsanity.com/paper/2610.16273/) (2026)

## Abstract (arXiv)

Encouraged by the success of deep neural networks on a variety of visual tasks, much theoretical and experimental work has been aimed at understanding and interpreting how vision networks operate. Meanwhile, deep neural networks have also achieved impressive performance in audio processing applications, both as sub-components of larger systems and as complete end-to-end systems by themselves. Despite their empirical successes, comparatively little is understood about how these audio models accomplish these tasks. In this work, we employ a recently developed statistical mechanical theory that connects geometric properties of network representations and the separability of classes to probe how information is untangled within neural networks trained to recognize speech. We observe that speaker-specific nuisance variations are discarded by the network's hierarchy, whereas task-relevant properties such as words and phonemes are untangled in later layers. Higher level concepts such as parts-of-speech and context dependence also emerge in the later layers of the network. Finally, we find that the deep representations carry out significant temporal untangling by efficiently extracting task-relevant features at each time step of the computation. Taken together, these findings shed light on how deep auditory models process time dependent input signals to achieve invariant speech recognition, and show how different concepts emerge through the layers of the network.