End-to-end speech recognition modeling from de-identified data
Not on the map. Why?
Abstract (arXiv)
De-identification of data used for automatic speech recognition modeling is a critical component in protecting privacy, especially in the medical domain. However, simply removing all personally identifiable information (PII) from end-to-end model training data leads to a significant performance degradation in particular for the recognition of names, dates, locations, and words from similar categories. We propose and evaluate a two-step method for partially recovering this loss. First, PII is identified, and each occurrence is replaced with a random word sequence of the same category. Then, corresponding audio is produced via text-to-speech or by splicing together matching audio fragments extracted from the corpus. These artificial audio/label pairs, together with speaker turns from the original data without PII, are used to train models. We evaluate the performance of this method on in-house data of medical conversations and observe a recovery of almost the entire performance degradation in the general word error rate while still maintaining a strong diarization performance. Our main focus is the improvement of recall and precision in the recognition of PII-related words. Depending on the PII category, between 50% - 90% of the performance degradation can be recovered using our proposed method.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 12 Jul 2022.
Builds on
17 of 25 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
- 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
- 2016De-identification of Patient Notes with Recurrent Neural Networkscited by 88
- 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
- 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
5 more
- 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
- 2021On Addressing Practical Challenges for RNN-Transducercited by 19
- 2020Privacy Guarantees for De-identifying Text Transformationscited by 8
- 2018A Deep Learning Architecture for De-identification of Patient Notes: Implementation and Evaluationcited by 6
- 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.