mlsanityData to 7 Oct 2026

SpeechPrompt v2: Prompt Tuning for Speech Classification Tasks

arXiv 2303.00733 ↗First posted 1 Mar 2023eess.AS cs.AI cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

Prompt tuning is a technology that tunes a small set of parameters to steer a pre-trained language model (LM) to directly generate the output for downstream tasks. Recently, prompt tuning has demonstrated its storage and computation efficiency in both natural language processing (NLP) and speech processing fields. These advantages have also revealed prompt tuning as a candidate approach to serving pre-trained LM for multiple tasks in a unified manner. For speech processing, SpeechPrompt shows its high parameter efficiency and competitive performance on a few speech classification tasks. However, whether SpeechPrompt is capable of serving a large number of tasks is unanswered. In this work, we propose SpeechPrompt v2, a prompt tuning framework capable of performing a wide variety of speech classification tasks, covering multiple languages and prosody-related tasks. The experiment result shows that SpeechPrompt v2 achieves performance on par with prior works with less than 0.15M trainable parameters in a unified framework.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

25 of 33 references.

References · 25

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  6. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  7. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  8. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  9. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 191
  10. 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 174
  11. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
  12. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
13 more
  1. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  2. 2021ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speechcited by 56
  3. 2021Universal Paralinguistic Speech Representations Using Self-Supervised Conformerscited by 32
  4. 2022A Multimodal Corpus for Emotion Recognition in Sarcasmcited by 29
  5. 2022WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Modelscited by 25
  6. 2021Learning Efficient Representations for Keyword Spotting with Triplet Losscited by 24
  7. 2021VoxCeleb Enrichment for Age and Gender Recognitioncited by 20
  8. 2022SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Taskscited by 19
  9. 2020AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognitioncited by 17
  10. 2020Improving End-to-End Speech-to-Intent Classification with Reptilecited by 17
  11. 2020MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detectioncited by 12
  12. 2022Structured Prompt Tuningcited by 4
  13. 2022Finstreder: Simple and fast Spoken Language Understanding with Finite State Transducers using modern Speech-to-Text modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 9; 2025: 5; 2026: 1.

31 papers.

Sort:

By area:

  1. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 123
  2. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 99
  3. 2023Joint Audio and Speech Understandingcited by 99
  4. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  5. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  6. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  7. 2023Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understandingcited by 21
  8. 2023SpeechGen: Unlocking the Generative Power of Speech Language Models with Promptscited by 21
  9. 2023Extending Whisper with prompt tuning to target-speaker ASRcited by 14
  10. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  11. 2024Decoder-only Architecture for Streaming End-to-end Speech Recognitioncited by 12
  12. 2023Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognitioncited by 10
19 more
  1. 2024Efficient Streaming LLM for Speech Recognitioncited by 10
  2. 2023UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructionscited by 9
  3. 2023A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Modelcited by 8
  4. 2024Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networkscited by 8
  5. 2023Instruction-Following Speech Recognitioncited by 7
  6. 2023Exploring In-Context Learning of Textless Speech Language Model for Speech Classification Taskscited by 5
  7. 2023Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Modelcited by 3
  8. 2025LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMscited by 3
  9. 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
  10. 2024Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Modelcited by 2
  11. 2024TokenVerse: Towards Unifying Speech and NLP Tasks via Transducer-based ASRcited by 2
  12. 2023Differentially Private Adapters for Parameter Efficient Acoustic Modelingcited by 1
  13. 2025UniPET-SPK: A Unified Framework for Parameter-Efficient Tuning of Pre-trained Speech Models for Robust Speaker Verificationcited by 1
  14. 2023CPPF: A contextual and post-processing-free model for automatic speech recognitioncited by 0
  15. 2024Developing an End-to-End Framework for Predicting the Social Communication Severity Scores of Children with Autism Spectrum Disordercited by 0
  16. 2024NapTune: Efficient Model Tuning for Mood Classification using Previous Night's Sleep Measures along with Wearable Time-seriescited by 0
  17. 2024Unified Pathological Speech Analysis with Prompt Tuningcited by 0
  18. 2025Conditional Information Bottleneck for Multimodal Fusion: Overcoming Shortcut Learning in Sarcasm Detectioncited by 0
  19. 2025TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activationcited by 0