mlsanityData to 14 Oct 2026

Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding

arXiv 2305.13512 ↗First posted 22 May 2023cs.CL cs.AI cs.SDcited by 21

Not on the map. Why?

Abstract (arXiv)

Recently, large pretrained language models have demonstrated strong language understanding capabilities. This is particularly reflected in their zero-shot and in-context learning abilities on downstream tasks through prompting. To assess their impact on spoken language understanding (SLU), we evaluate several such models like ChatGPT and OPT of different sizes on multiple benchmarks. We verify the emergent ability unique to the largest models as they can reach intent classification accuracy close to that of supervised models with zero or few shots on various languages given oracle transcripts. By contrast, the results for smaller models fitting a single GPU fall far behind. We note that the error cases often arise from the annotation scheme of the dataset; responses from ChatGPT are still reasonable. We show, however, that the model is worse at slot filling, and its performance is sensitive to ASR errors, suggesting serious challenges for the application of those textual models on SLU.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

37 of 47 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
25 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  6. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  8. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  9. 2022Prompting GPT-3 To Be Reliablecited by 260
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  11. 2018Towards end-to-end spoken language understandingcited by 92
  12. 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
  13. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  14. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
  15. 2021A Survey on Spoken Language Understanding: Recent Advances and New Frontierscited by 53
  16. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  17. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  18. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 32
  19. 2022WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Modelscited by 26
  20. 2021Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understandingcited by 22
  21. 2022XTREME-S: Evaluating Cross-lingual Speech Representationscited by 22
  22. 2021End-to-End Spoken Language Understanding for Generalized Voice Assistantscited by 17
  23. 2022Two-Pass Low Latency End-to-End Spoken Language Understandingcited by 8
  24. 2022A Unified Framework for Multi-intent Spoken Language Understanding with promptingcited by 3
  25. 2022Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding taskscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 7; 2026: 2.

21 papers.

Sort:

By area:

  1. 2024Large Language Models Meet NLP: A Surveycited by 103
  2. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  3. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 54
  4. 2023Can Whisper perform speech-based in-context learning?cited by 35
  5. 2023Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learningcited by 29
  6. 2025Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMscited by 29
  7. 2024Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyondcited by 18
  8. 2023Leveraging Large Language Models for Exploiting ASR Uncertaintycited by 15
  9. 2024Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasetscited by 13
  10. 2024Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networkscited by 8
  11. 2026CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systemscited by 6
  12. 2024Towards interfacing large language models with ASR systems using confidence measures and promptingcited by 4
9 more
  1. 2023Speech Understanding on Tiny Devices with A Learning Cachecited by 3
  2. 2024CroPrompt: Cross-task Interactive Prompting for Zero-shot Spoken Language Understandingcited by 3
  3. 2025Spoken Language Understanding on Unseen Tasks With In-Context Learningcited by 2
  4. 2025Do Large Language Models Need Intent? Revisiting Response Generation Strategies for Service Assistantcited by 2
  5. 2025MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmarkcited by 2
  6. 2025Leveraging Information Retrieval to Enhance Spoken Language Understanding Prompts in Few-Shot Learningcited by 0
  7. 2025Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missionscited by 0
  8. 2025A Computational Approach to Modeling Conversational Systems: Analyzing Large-Scale Quasi-Patterned Dialogue Flowscited by 0
  9. 2026Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecturecited by 0