mlsanityData to 13 Oct 2026

Detecting Unintended Memorization in Language-Model-Fused ASR

arXiv 2204.09606 ↗First posted 20 Apr 2022cs.CL cs.CR cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

End-to-end (E2E) models are often being accompanied by language models (LMs) via shallow fusion for boosting their overall quality as well as recognition of rare words. At the same time, several prior works show that LMs are susceptible to unintentionally memorizing rare or unique sequences in the training data. In this work, we design a framework for detecting memorization of random textual sequences (which we call canaries) in the LM training data when one has only black-box (query) access to LM-fused speech recognizer, as opposed to direct access to the LM. On a production-grade Conformer RNN-T E2E model fused with a Transformer LM, we show that detecting memorization of singly-occurring canaries from the LM training data of 300M examples is possible. Motivated to protect privacy, we also show that such memorization gets significantly reduced by per-example gradient-clipped LM training without compromising overall quality.

Key facts. Cited by 9 papers in the map (data through 13 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 20 Apr 2022.

Builds on

25 of 35 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,106
  2. 2016Deep Learning with Differential Privacycited by 2,718
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  4. 2020Extracting Training Data from Large Language Modelscited by 2,181
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,844
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,359
  7. 2021Membership Inference Attacks From First Principlescited by 872
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  10. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 456
  11. 2020Label-Only Membership Inference Attackscited by 386
  12. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 215
13 more
  1. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  2. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  3. 2020Hybrid Autoregressive Transducer (hat)cited by 107
  4. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 104
  5. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 100
  6. 2020Training Production Language Models without Memorizing User Datacited by 68
  7. 2020Understanding Unintended Memorization in Federated Learningcited by 36
  8. 2021Tied & Reduced RNN-T Decodercited by 35
  9. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  10. 2021Revealing and Protecting Labels in Distributed Trainingcited by 17
  11. 2022Differentially Private Speaker Anonymizationcited by 16
  12. 2021Lookup-Table Recurrent Language Models for Long Tail Speech Recognitioncited by 9
  13. 2021A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter Itcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5; 2024: 1.

9 papers.

  1. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  2. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  3. 2023Recovering from Privacy-Preserving Masking with Large Language Modelscited by 9
  4. 2023A Note On Interpreting Canary Exposurecited by 6
  5. 2023Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemblecited by 6
  6. 2023Unintended Memorization in Large ASR Models, and How to Mitigate Itcited by 5
  7. 2023Quantifying Overfitting: Evaluating Neural Network Performance through Analysis of Null Spacecited by 4
  8. 2022Mitigating Unintended Memorization in Language Models via Alternating Teachingcited by 3
  9. 2024Noise Masking Attacks and Defenses for Pretrained Speech Modelscited by 3