mlsanityData to 14 Oct 2026

Model Extraction Attack against Self-supervised Speech Models

arXiv 2211.16044 ↗First posted 29 Nov 2022cs.SD cs.CL cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) speech models generate meaningful representations of given clips and achieve incredible performance across various downstream tasks. Model extraction attack (MEA) often refers to an adversary stealing the functionality of the victim model with only query access. In this work, we study the MEA problem against SSL speech model with a small number of queries. We propose a two-stage framework to extract the model. In the first stage, SSL is conducted on the large-scale unlabeled corpus to pre-train a small speech model. Secondly, we actively sample a small portion of clips from the unlabeled corpus and query the target model with these clips to acquire their representations as labels for the small model's second-stage training. Experiment results show that our sampling methods can effectively extract the target model without knowing any information about its model architecture.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 29 Nov 2022.

Builds on

22 of 34 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  9. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  10. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  11. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  12. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
10 more
  1. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 193
  2. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  3. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 144
  4. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 139
  5. 2018Stealing Hyperparameters in Machine Learningcited by 118
  6. 2018Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Datacited by 70
  7. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 53
  8. 2021Model Stealing Attacks Against Inductive Graph Neural Networkscited by 50
  9. 2020Model Extraction Attacks on Graph Neural Networks: Taxonomy and Realizationcited by 40
  10. 2021Killing One Bird with Two Stones: Model Extraction and Attribute Inference Attacks against BERT-based APIscited by 8

Built on by

Not cited yet.