# From `Snippet-lects' to Doculects and Dialects: Leveraging Neural Representations of Speech for Placing Audio Signals in a Language Landscape

arXiv: https://arxiv.org/abs/2305.18602 · HTML: https://mlsanity.com/paper/2305.18602/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 14 of 14)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [VoxLingua107: a Dataset for Spoken Language Recognition](https://mlsanity.com/paper/2011.12998/) (2020)
- [Exploring wav2vec 2.0 on speaker verification and language identification](https://mlsanity.com/paper/2012.06185/) (2020)
- [Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing](https://mlsanity.com/paper/1807.00914/) (2018)
- [Probing phoneme, language and speaker information in unsupervised speech representations](https://mlsanity.com/paper/2203.16193/) (2022)
- [Improved Language Identification Through Cross-Lingual Self-Supervised Learning](https://mlsanity.com/paper/2107.04082/) (2021)
- [Neural Representations for Modeling Variation in Speech](https://mlsanity.com/paper/2011.12649/) (2020)
- [Probabilistic Typology: Deep Generative Models of Vowel Inventories](https://mlsanity.com/paper/1705.01684/) (2017)
- [Leveraging pre-trained representations to improve access to untranscribed speech from endangered languages](https://mlsanity.com/paper/2103.14583/) (2021)
- [Tracking Typological Traits of Uralic Languages in Distributed Language Representations](https://mlsanity.com/paper/1711.05468/) (2017)
- [HUBERT Untangles BERT to Improve Transfer across NLP Tasks](https://mlsanity.com/paper/1910.12647/) (2019)

## Built on by (most cited first, 1 of 1)

- [Predicting positive transfer for improved low-resource speech recognition using acoustic pseudo-tokens](https://mlsanity.com/paper/2402.02302/) (2024)

## Abstract (arXiv)

XLSR-53 a multilingual model of speech, builds a vector representation from audio, which allows for a range of computational treatments. The experiments reported here use this neural representation to estimate the degree of closeness between audio files, ultimately aiming to extract relevant linguistic properties. We use max-pooling to aggregate the neural representations from a "snippet-lect" (the speech in a 5-second audio snippet) to a "doculect" (the speech in a given resource), then to dialects and languages. We use data from corpora of 11 dialects belonging to 5 less-studied languages. Similarity measurements between the 11 corpora bring out greatest closeness between those that are known to be dialects of the same language. The findings suggest that (i) dialect/language can emerge among the various parameters characterizing audio files and (ii) estimates of overall phonetic/phonological closeness can be obtained for a little-resourced or fully unknown language. The findings help shed light on the type of information captured by neural representations of speech and how it can be extracted from these representations