# What Can an Accent Identifier Learn? Probing Phonetic and Prosodic Information in a Wav2vec2-based Accent Identification Model arXiv: https://arxiv.org/abs/2306.06524 ยท HTML: https://mlsanity.com/paper/2306.06524/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Builds on 22 of its 43 references that are in the map. - First posted on arXiv 10 Jun 2023. ## Builds on (most cited first, 22 of 22) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021) - [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015) - [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021) - [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021) - [Exploring wav2vec 2.0 on speaker verification and language identification](https://mlsanity.com/paper/2012.06185/) (2020) - [Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations](https://mlsanity.com/paper/2110.14513/) (2021) - [ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers](https://mlsanity.com/paper/2204.09224/) (2022) - [On the Utility of Self-supervised Models for Prosody-related Tasks](https://mlsanity.com/paper/2210.07185/) (2022) - [Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems](https://mlsanity.com/paper/1709.04482/) (2017) - [Probing Acoustic Representations for Phonetic Properties](https://mlsanity.com/paper/2010.13007/) (2020) - [The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methods](https://mlsanity.com/paper/2102.10233/) (2021) - [Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learning](https://mlsanity.com/paper/2109.07349/) (2021) - [Probing Speech Emotion Recognition Transformers for Linguistic Knowledge](https://mlsanity.com/paper/2204.00400/) (2022) - [Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment](https://mlsanity.com/paper/2203.15937/) (2022) - [Speech-based emotion recognition with self-supervised models using attentive channel-wise correlations and label smoothing](https://mlsanity.com/paper/2211.01756/) (2022) - [Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition](https://mlsanity.com/paper/1803.04567/) (2018) - [Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Information](https://mlsanity.com/paper/2205.03759/) (2022) - [Deep Learning For Prominence Detection In Children's Read Speech](https://mlsanity.com/paper/2110.14273/) (2021) - [Accent Recognition with Hybrid Phonetic Features](https://mlsanity.com/paper/2105.01920/) (2021) ## Built on by (most cited first, 18 of 18) - [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023) - [A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models](https://mlsanity.com/paper/2408.13678/) (2024) - [VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing](https://mlsanity.com/paper/2404.06674/) (2024) - [Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0](https://mlsanity.com/paper/2508.08110/) (2025) - [PRiSM: Benchmarking Phone Realization in Speech Models](https://mlsanity.com/paper/2601.14046/) (2026) - [Different Speech Translation Models Encode and Translate Speaker Gender Differently](https://mlsanity.com/paper/2506.02172/) (2025) - [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025) - [An Exploration of Mamba for Speech Self-Supervised Models](https://mlsanity.com/paper/2506.12606/) (2025) - [Accent Vector: Controllable Accent Manipulation for Multilingual TTS Without Accented Data](https://mlsanity.com/paper/2603.07534/) (2026) - [Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model](https://mlsanity.com/paper/2405.01730/) (2024) - [Representation of perceived prosodic similarity of conversational feedback](https://mlsanity.com/paper/2505.13268/) (2025) - [Tone recognition in low-resource languages of North-East India: peeling the layers of SSL-based speech models](https://mlsanity.com/paper/2506.03606/) (2025) - [Probing for Phonology in Self-Supervised Speech Representations: A Case Study on Accent Perception](https://mlsanity.com/paper/2506.17542/) (2025) - [Emphasis Sensitivity in Speech Representations](https://mlsanity.com/paper/2508.11566/) (2025) - [Geolocation-Aware Robust Spoken Language Identification](https://mlsanity.com/paper/2508.17148/) (2025) - [ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition](https://mlsanity.com/paper/2603.03359/) (2026) - [Perceptual compensation for tonal context in self-supervised speech models](https://mlsanity.com/paper/2606.17835/) (2026) - [Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English](https://mlsanity.com/paper/2606.23948/) (2026) ## Abstract (arXiv) This study is focused on understanding and quantifying the change in phoneme and prosody information encoded in the Self-Supervised Learning (SSL) model, brought by an accent identification (AID) fine-tuning task. This problem is addressed based on model probing. Specifically, we conduct a systematic layer-wise analysis of the representations of the Transformer layers on a phoneme correlation task, and a novel word-level prosody prediction task. We compare the probing performance of the pre-trained and fine-tuned SSL models. Results show that the AID fine-tuning task steers the top 2 layers to learn richer phoneme and prosody representation. These changes share some similarities with the effects of fine-tuning with an Automatic Speech Recognition task. In addition, we observe strong accent-specific phoneme representations in layer 9. To sum up, this study provides insights into the understanding of SSL features and their interactions with fine-tuning tasks.