# The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Services

arXiv: https://arxiv.org/abs/1905.07082 · HTML: https://mlsanity.com/paper/1905.07082/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 25 of 25)

- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Hidden Backdoors in Human-Centric Language Models](https://mlsanity.com/paper/2105.00164/) (2021)
- [Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy](https://mlsanity.com/paper/2405.14800/) (2024)
- [User Inference Attacks on Large Language Models](https://mlsanity.com/paper/2310.09266/) (2023)
- [Fine-Tuning Large Language Models with User-Level Differential Privacy](https://mlsanity.com/paper/2407.07737/) (2024)
- [User-Level Membership Inference Attack against Metric Embedding Learning](https://mlsanity.com/paper/2203.02077/) (2022)
- [Does CLIP Know My Face?](https://mlsanity.com/paper/2209.07341/) (2022)
- [FACE-AUDITOR: Data Auditing in Facial Recognition Systems](https://mlsanity.com/paper/2304.02782/) (2023)
- [A General Framework for Data-Use Auditing of ML Models](https://mlsanity.com/paper/2407.15100/) (2024)
- [SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems](https://mlsanity.com/paper/2309.07983/) (2023)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Data Isotopes for Data Provenance in DNNs](https://mlsanity.com/paper/2208.13893/) (2022)
- [l-Leaks: Membership Inference Attacks with Logits](https://mlsanity.com/paper/2205.06469/) (2022)
- [Attesting Distributional Properties of Training Data for Machine Learning](https://mlsanity.com/paper/2308.09552/) (2023)
- [Anonymity Unveiled: A Practical Framework for Auditing Data Use in Deep Learning Models](https://mlsanity.com/paper/2409.06280/) (2024)
- [Active Membership Inference Test (aMINT): Enhancing Model Auditability with Multi-Task Learning](https://mlsanity.com/paper/2509.07879/) (2025)
- [Data Origin Inference in Machine Learning](https://mlsanity.com/paper/2211.13416/) (2022)
- [Improving Membership Inference in ASR Model Auditing with Perturbed Loss Features](https://mlsanity.com/paper/2405.01207/) (2024)
- [Fingerprint Membership and Identity Inference Against Generative Adversarial Networks](https://mlsanity.com/paper/2406.15253/) (2024)
- [CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning](https://mlsanity.com/paper/2505.19119/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [Assessing the Effectiveness of Membership Inference on Generative Music](https://mlsanity.com/paper/2512.21762/) (2025)
- [Private Training Set Inspection in MLaaS](https://mlsanity.com/paper/2305.09058/) (2023)
- [Robust Federated Learning via Byzantine Filtering over Encrypted Updates](https://mlsanity.com/paper/2602.05410/) (2026)
- [Membership Inference for Contrastive Pre-training Models with Text-only PII Queries](https://mlsanity.com/paper/2603.14222/) (2026)

## Abstract (arXiv)

With the rapid development of deep learning techniques, the popularity of voice services implemented on various Internet of Things (IoT) devices is ever increasing. In this paper, we examine user-level membership inference in the problem space of voice services, by designing an audio auditor to verify whether a specific user had unwillingly contributed audio used to train an automatic speech recognition (ASR) model under strict black-box access. With user representation of the input audio data and their corresponding translated text, our trained auditor is effective in user-level audit. We also observe that the auditor trained on specific data can be generalized well regardless of the ASR model architecture. We validate the auditor on ASR models trained with LSTM, RNNs, and GRU algorithms on two state-of-the-art pipelines, the hybrid ASR system and the end-to-end ASR system. Finally, we conduct a real-world trial of our auditor on iPhone Siri, achieving an overall accuracy exceeding 80%. We hope the methodology developed in this paper and findings can inform privacy advocates to overhaul IoT privacy.