# How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings

arXiv: https://arxiv.org/abs/2109.10179 · HTML: https://mlsanity.com/paper/2109.10179/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [Similarity Analysis of Contextual Word Representation Models](https://mlsanity.com/paper/2005.01172/) (2020)
- [Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains](https://mlsanity.com/paper/1906.01539/) (2019)
- [Correlating neural and symbolic representations of language](https://mlsanity.com/paper/1905.06401/) (2019)
- [Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models](https://mlsanity.com/paper/1811.00403/) (2018)
- [Deep convolutional acoustic word embeddings using word-pair side information](https://mlsanity.com/paper/1510.01032/) (2015)
- [Encoding of phonology in a recurrent neural model of grounded speech](https://mlsanity.com/paper/1706.03815/) (2017)
- [Similarity Analysis of Self-Supervised Speech Representations](https://mlsanity.com/paper/2010.11481/) (2020)
- [Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches](https://mlsanity.com/paper/1611.02550/) (2016)
- [Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis](https://mlsanity.com/paper/2011.12073/) (2020)
- [What do Language Representations Really Represent?](https://mlsanity.com/paper/1901.02646/) (2019)
- [Analyzing analytical methods: The case of phonology in neural models of spoken language](https://mlsanity.com/paper/2004.07070/) (2020)
- [Higher-order Comparisons of Sentence Encoder Representations](https://mlsanity.com/paper/1909.00303/) (2019)
- [Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study](https://mlsanity.com/paper/2106.08686/) (2021)
- [Analyzing autoencoder-based acoustic word embeddings](https://mlsanity.com/paper/2004.01647/) (2020)
- [Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language](https://mlsanity.com/paper/2106.12834/) (2021)
- [Evaluating computational models of infant phonetic learning across languages](https://mlsanity.com/paper/2008.02888/) (2020)
- [A phonetic model of non-native spoken word processing](https://mlsanity.com/paper/2101.11332/) (2021)
- [Learning to Understand Child-directed and Adult-directed Speech](https://mlsanity.com/paper/2005.02721/) (2020)

## Built on by (most cited first, 8 of 8)

- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [Encoding of lexical tone in self-supervised models of spoken language](https://mlsanity.com/paper/2403.16865/) (2024)
- [Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning](https://mlsanity.com/paper/2204.05148/) (2022)
- [Analyzing the Representational Geometry of Acoustic Word Embeddings](https://mlsanity.com/paper/2301.03012/) (2023)
- [Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations](https://mlsanity.com/paper/2403.08738/) (2024)
- [Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings](https://mlsanity.com/paper/2209.06633/) (2022)
- [A Technique for Isolating Lexically-Independent Phonetic Dependencies in Generative CNNs](https://mlsanity.com/paper/2506.09218/) (2025)
- [Lost but not erased: Finding traces of a forgotten language in neural speech models](https://mlsanity.com/paper/2608.25976/) (2026)

## Abstract (arXiv)

How do neural networks "perceive" speech sounds from unknown languages? Does the typological similarity between the model's training language (L1) and an unknown language (L2) have an impact on the model representations of L2 speech signals? To answer these questions, we present a novel experimental design based on representational similarity analysis (RSA) to analyze acoustic word embeddings (AWEs) -- vector representations of variable-duration spoken-word segments. First, we train monolingual AWE models on seven Indo-European languages with various degrees of typological similarity. We then employ RSA to quantify the cross-lingual similarity by simulating native and non-native spoken-word processing using AWEs. Our experiments show that typological similarity indeed affects the representational similarity of the models in our study. We further discuss the implications of our work on modeling speech processing and language similarity with neural networks.