# Supervised Acoustic Embeddings And Their Transferability Across Languages

arXiv: https://arxiv.org/abs/2301.01020 · HTML: https://mlsanity.com/paper/2301.01020/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech](https://mlsanity.com/paper/1803.08976/) (2018)
- [Learning Robust and Multilingual Speech Representations](https://mlsanity.com/paper/2001.11128/) (2020)
- [Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder](https://mlsanity.com/paper/1603.00982/) (2016)
- [The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition](https://mlsanity.com/paper/1609.05625/) (2016)
- [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021)
- [Neural Representations for Modeling Variation in Speech](https://mlsanity.com/paper/2011.12649/) (2020)
- [A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings](https://mlsanity.com/paper/2012.02221/) (2020)
- [A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings](https://mlsanity.com/paper/2012.07387/) (2020)
- [Multilingual acoustic word embedding models for processing zero-resource languages](https://mlsanity.com/paper/2002.02109/) (2020)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)

## Built on by (most cited first, 1 of 1)

- [ArTST: Arabic Text and Speech Transformer](https://mlsanity.com/paper/2310.16621/) (2023)

## Abstract (arXiv)

In speech recognition, it is essential to model the phonetic content of the input signal while discarding irrelevant factors such as speaker variations and noise, which is challenging in low-resource settings. Self-supervised pre-training has been proposed as a way to improve both supervised and unsupervised speech recognition, including frame-level feature representations and Acoustic Word Embeddings (AWE) for variable-length segments. However, self-supervised models alone cannot learn perfect separation of the linguistic content as they are trained to optimize indirect objectives. In this work, we experiment with different pre-trained self-supervised features as input to AWE models and show that they work best within a supervised framework. Models trained on English can be transferred to other languages with no adaptation and outperform self-supervised models trained solely on the target languages.