mlsanityData to 19 Oct 2026

Towards Privacy-Preserving ASR: Speaker-Level Machine Unlearning

arXiv 2610.16196 ↗First posted 12 Oct 2026eess.AScited by 0

On the map: Speech and audio language models → Speech recognition models and training also fits: Anti-spoofing and adversarial audio attacks

Abstract (arXiv)

Automatic Speech Recognition (ASR) systems are commonly designed to transcribe linguistic content independently of speaker identity. However, speaker-specific information can persist within the latent representations of ASR models, raising privacy concerns when training data includes sensitive individuals. While machine unlearning has been studied in several speech tasks, applying speaker unlearning to ASR remains challenging due to the strong entanglement of linguistic and speaker-related information in speech representations. In this work, we conduct a layer-wise analysis of Self-Supervised Learning (SSL) based ASR representations, revealing that speaker identity is primarily encoded in intermediate encoder layers. Based on this observation, we explore a feature-level unlearning strategy that targets these identity structures. Experiments on the VCTK corpus show that the proposed approach removes the influence of specific speakers while maintaining robust transcription performance.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech recognition models and training, one of 11 directions in Speech and audio language models. Builds on 24 of its 33 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

24 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025"Alexa, can you forget me?" Machine Unlearning Benchmark in Spoken Language Understanding21 May
  2. 2023Unintended Memorization in Large ASR Models, and How to Mitigate It18 Oct
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing26 Oct

Other references · 21

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,289
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 870
  4. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 621
  5. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 398
  6. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearningcited by 280
  7. 2023Towards Unbounded Machine Unlearningcited by 272
  8. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 197
  9. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignmentcited by 102
  10. 2024Machine Unlearning in Generative AI: A Surveycited by 54
  11. 2022Privacy Adhering Machine Un-learning in NLPcited by 37
  12. 2019The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Servicescited by 26
9 more
  1. 2021Membership Inference Attacks Against Self-supervised Speech Modelscited by 19
  2. 2023DUCK: Distance-based Unlearning via Centroid Kinematicscited by 14
  3. 2021A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter Itcited by 10
  4. 2025Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalizationcited by 8
  5. 2025Speech Unlearningcited by 8
  6. 2025Do Not Mimic My Voice: Speaker Identity Unlearning for Zero-Shot Text-to-Speechcited by 8
  7. 2024Fine-Tuning Automatic Speech Recognition for People with Parkinson's: An Effective Strategy for Enhancing Speech Technology Accessibilitycited by 7
  8. 2024Improving Membership Inference in ASR Model Auditing with Perturbed Loss Featurescited by 4
  9. 2025Breaking the Transcription Bottleneck: Fine-tuning ASR Models for Extremely Low-Resource Fieldwork Languagescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Machine Unlearning in ASR
  2. 2023Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clustering
  3. 2022Language-Independent Speaker Anonymization Approach using Self-Supervised Pre-Trained Models
  4. 2023On-Device Speaker Anonymization of Acoustic Embeddings for ASR based onFlexible Location Gradient Reversal Layer
  5. 2022Analyzing the factors affecting usefulness of Self-Supervised Pre-trained Representations for Speech Recognition
  6. 2024Training Large ASR Encoders with Differential Privacy
  7. 2026Speaker Group Encoding in Self-supervised Speech Recognition Models
  8. 2026Continual Speaker Identity Unlearning with Minimal Interference