mlsanityData to 14 Oct 2026

Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining

arXiv 2410.18371 ↗First posted 24 Oct 2024cs.SD cs.AI eess.AScited by 10

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Audio can disclose PII, particularly when combined with related text data. Therefore, it is essential to develop tools to detect privacy leakage in Contrastive Language-Audio Pretraining(CLAP). Existing MIAs need audio as input, risking exposure of voiceprint and requiring costly shadow models. We first propose PRMID, a membership inference detector based probability ranking given by CLAP, which does not require training shadow models but still requires both audio and text of the individual as input. To address these limitations, we then propose USMID, a textual unimodal speaker-level membership inference detector, querying the target model using only text data. We randomly generate textual gibberish that are clearly not in training dataset. Then we extract feature vectors from these texts using the CLAP model and train a set of anomaly detectors on them. During inference, the feature vector of each test text is input into the anomaly detector to determine if the speaker is in the training set (anomalous) or not (normal). If available, USMID can further enhance detection by integrating real audio of the tested speaker. Extensive experiments on various CLAP model architectures and datasets demonstrate that USMID outperforms baseline methods using only text data.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 18 of its 36 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

18 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study29 Sep
  2. 2023SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems14 Sep
  3. 2022M^4I: Multi-modal Models Membership Inference15 Sep
  4. 2021Membership Inference Attacks Against Self-supervised Speech Models9 Nov

Other references · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  6. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  7. 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 176
  8. 2021Multimodal Contrastive Training for Visual Representation Learningcited by 124
  9. 2013One-Class Classification: Taxonomy of Study and Review of Techniquescited by 68
  10. 2019Hear "No Evil", See "Kenansville": Efficient and Transferable Black-Box Attacks on Speech Recognition and Voice Identification Systemscited by 38
  11. 2024Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learningcited by 21
  12. 2022Does CLIP Know My Face?cited by 19
2 more
  1. 2024Advancing Multi-grained Alignment for Contrastive Language-Audio Pre-trainingcited by 14
  2. 2022User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 2.

10 papers.

  1. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  2. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  3. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  4. 2025A Reward-driven Automated Webshell Malicious-code Generator for Red-teamingcited by 1
  5. 2025Assessing the Effectiveness of Membership Inference on Generative Musiccited by 1
  6. 2025DMRL: Data- and Model-aware Reward Learning for Data Extractioncited by 0
  7. 2025VisuoAlign: Safety Alignment of LVLMs with Multimodal Tree Searchcited by 0
  8. 2025Information-Dense Reasoning for Efficient and Auditable Security Alert Triagecited by 0
  9. 2026ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defensecited by 0
  10. 2026From Interpretability to Control: Insights from Six Years of the TrustNLP Workshopcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Membership Inference Attacks against Large Audio Language Models
  2. 2026Membership Inference for Contrastive Pre-training Models with Text-only PII Queries
  3. 2026Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models
  4. 2026Black-Box Membership Inference via Word-Level Probability Estimation
  5. 2026HearSay Benchmark: Do Audio LLMs Leak What They Hear?
  6. 2025VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks
  7. 2025The Man Behind the Sound: Demystifying Audio Private Attribute Profiling via Multimodal Large Language Model Agents
  8. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector