mlsanityData to 7 Oct 2026

Exploring In-Context Learning of Textless Speech Language Model for Speech Classification Tasks

arXiv 2310.12477 ↗First posted 19 Oct 2023eess.AS cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Ever since the development of GPT-3 in the natural language processing (NLP) field, in-context learning (ICL) has played an essential role in utilizing large language models (LLMs). By presenting the LM utterance-label demonstrations at the input, the LM can accomplish few-shot learning without relying on gradient descent or requiring explicit modification of its parameters. This enables the LM to perform various downstream tasks in a black-box manner. Despite the success of ICL in NLP, little work is exploring the possibility of ICL in speech processing. This study is the first work exploring ICL for speech classification tasks with textless speech LM. We first show that the current speech LM lacks the ICL capability. We then perform warmup training on the speech LM, equipping the LM with demonstration learning capability. This paper explores and proposes the first speech LM capable of performing unseen classification tasks in an ICL manner.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

27 of 36 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  7. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  8. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  10. 2021MetaICL: Learning to Learn In Contextcited by 450
  11. 2023Larger language models do in-context learning differentlycited by 386
  12. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
15 more
  1. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  2. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  3. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  4. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  5. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  6. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
  7. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  8. 2018Neural Voice Cloning with a Few Samplescited by 100
  9. 2023Textually Pretrained Speech Language Modelscited by 99
  10. 2022Understanding and Improving Visual Prompting: A Label-Mapping Perspectivecited by 86
  11. 2020Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resourcescited by 83
  12. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 56
  13. 2023Pre-Training to Learn in Contextcited by 41
  14. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 31
  15. 2023Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Modelcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
  2. 2024LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generationcited by 4
  3. 2025Spoken Language Understanding on Unseen Tasks With In-Context Learningcited by 2
  4. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Modelscited by 2
  5. 2026In-Context Adaptation of Encoder-Decoder Models in Speech Recognitioncited by 0