# Exploratory Evaluation of Speech Content Masking

arXiv: https://arxiv.org/abs/2401.03936 · HTML: https://mlsanity.com/paper/2401.03936/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 16 of 16)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019](https://mlsanity.com/paper/1905.11449/) (2019)
- [The GDPR & Speech Data: Reflections of Legal and Technology Communities, First Steps towards a Common Understanding](https://mlsanity.com/paper/1907.03458/) (2019)
- [Preech: A System for Privacy-Preserving Speech Transcription](https://mlsanity.com/paper/1909.04198/) (2019)
- [Speaker Conditional WaveRNN: Towards Universal Neural Vocoder for Unseen Speaker and Recording Conditions](https://mlsanity.com/paper/2008.05289/) (2020)
- [Learning Disentangled Phone and Speaker Representations in a Semi-Supervised VQ-VAE Paradigm](https://mlsanity.com/paper/2010.10727/) (2020)
- [Scene-Aware Audio Rendering via Deep Acoustic Analysis](https://mlsanity.com/paper/1911.06245/) (2019)
- [Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders](https://mlsanity.com/paper/2102.06610/) (2021)
- [Revisiting Speech Content Privacy](https://mlsanity.com/paper/2110.06760/) (2021)
- [Exploring Disentanglement with Multilingual and Monolingual VQ-VAE](https://mlsanity.com/paper/2105.01573/) (2021)
- [New Challenges for Content Privacy in Speech and Audio](https://mlsanity.com/paper/2301.08925/) (2023)
- [Adversarial Privacy Protection on Speech Enhancement](https://mlsanity.com/paper/2206.08170/) (2022)

## Built on by (most cited first, 3 of 3)

- [Content Anonymization for Privacy in Long-form Audio](https://mlsanity.com/paper/2510.12780/) (2025)
- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Is Word Error Rate Enough? Rethinking Privacy Evaluation in Speech with Entity-Aware Metrics](https://mlsanity.com/paper/2610.08831/) (2026)

## Abstract (arXiv)

Most recent speech privacy efforts have focused on anonymizing acoustic speaker attributes but there has not been as much research into protecting information from speech content. We introduce a toy problem that explores an emerging type of privacy called "content masking" which conceals selected words and phrases in speech. In our efforts to define this problem space, we evaluate an introductory baseline masking technique based on modifying sequences of discrete phone representations (phone codes) produced from a pre-trained vector-quantized variational autoencoder (VQ-VAE) and re-synthesized using WaveRNN. We investigate three different masking locations and three types of masking strategies: noise substitution, word deletion, and phone sequence reversal. Our work attempts to characterize how masking affects two downstream tasks: automatic speech recognition (ASR) and automatic speaker verification (ASV). We observe how the different masks types and locations impact these downstream tasks and discuss how these issues may influence privacy goals.