SLURP: A Spoken Language Understanding Resource Package
Not on the map. Why?
Abstract (arXiv)
Spoken Language Understanding infers semantic meaning directly from audio data, and thus promises to reduce error propagation and misunderstandings in end-user applications. However, publicly available SLU resources are limited. In this paper, we release SLURP, a new SLU package containing the following: (1) A new challenging dataset in English spanning 18 domains, which is substantially bigger and linguistically more diverse than existing datasets; (2) Competitive baselines based on state-of-the-art NLU and ASR systems; (3) A new transparent metric for entity labelling which enables a detailed error analysis for identifying potential areas of improvement. SLURP is available at https: //github.com/pswietojanski/slurp.
Key facts. Cited by 197 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Nov 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 10; 2022: 32; 2023: 50; 2024: 44; 2025: 33; 2026: 28.
197 papers.
Sort:
By area:
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
- 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
- 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
- 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
- 2023Textually Pretrained Speech Language Modelscited by 99
- 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
- 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
- 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
- 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 77
38 more
- 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
- 2024SpeechVerse: A Large-scale Generalizable Audio Language Modelcited by 62
- 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
- 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Settingcited by 47
- 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
- 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
- 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
- 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
- 2022LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Taggingcited by 29
- 2023Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understandingcited by 29
- 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 27
- 2022WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Modelscited by 25
- 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
- 2021Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understandingcited by 22
- 2022STOP: A dataset for Spoken Task Oriented Semantic Parsingcited by 22
- 2023Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understandingcited by 21
- 2023Can Contextual Biasing Remain Effective with Whisper and GPT-2?cited by 21
- 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
- 2024Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyondcited by 20
- 2024Improving Semantic Understanding in Speech Language Models via Brain-tuningcited by 20
- 2025AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluationcited by 20
- 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
- 2023NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Taskscited by 19
- 2024Exploring the Capability of Mamba in Speech Applicationscited by 19
- 2022A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understandingcited by 18
- 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18
- 2023CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Taskscited by 18
- 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
- 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 16
- 2022MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialoguecited by 15
- 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
- 2024Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversationcited by 14
- 2024Benchmarking Representations for Speech, Music, and Acoustic Eventscited by 14
- 2024Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognitioncited by 14
- 2025SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction Fine-Tuningcited by 14
- 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
50 most cited of 197 All 197 (JSON)
- 2026Talk2Agent: Benchmarking Voice Interfaces for Text Agents30 Sep 2026
- 2026EdgeCraft: Automated Model Crafting for Edge IoT28 Sep 2026
- 2026Quieter Than the Room: Representation Drift and Task Robustness in Speech Encoders23 Sep 2026
- 2026Same evidence, different judgments: Evidence noncommutative in vision/speech-text conflicts22 Sep 2026
- 2026Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs19 Sep 2026
- 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding10 Sep 2026
- 2026Topic Matching in the Wild: Benchmark and Lessons from Real-World ASR Transcripts27 Aug 2026
- 2026Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text24 Aug 2026
- 2026Aslema at NADI 2026: Data Augmentation for Intent Recognition and Slot Filling19 Aug 2026
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations4 Aug 2026
- 2026ESF-Bench: Benchmarking Challenging Slot-Filling Scenarios for Real-World Enterprise Applications25 Jul 2026
- 2026GigaAM Multilingual: Foundation Model for Underrepresented Languages11 Jul 2026
38 more
- 2026Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?24 Jun 2026
- 2026Selective Capability Unlearning in End-to-End Spoken Language Understanding23 Jun 2026
- 2026When Do LLMs Replace Fine-Tuned NLU? A Decision Framework for Intent Detection in Production Conversational Systems19 Jun 2026
- 2026CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agents19 Jun 2026
- 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects31 May 2026
- 2026A Unified and Reproducible Experimentation Framework for Speech Understanding29 May 2026
- 2026IntentGrasp: A Comprehensive Benchmark for Intent Understanding7 May 2026
- 2026StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario29 Apr 2026
- 2026Latent Preference Modeling for Multi-Session Personalized Tool Calling20 Apr 2026
- 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use17 Apr 2026
- 2026SLURP-TN : Resource for Tunisian Dialect Spoken Language Understanding23 Mar 2026
- 2026TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild23 Mar 2026
- 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models10 Mar 2026
- 2026STACodec: Semantic Token Assignment for Balancing Acoustic Fidelity and Semantic Information in Audio Codecs5 Feb 2026
- 2026Interpretability of the Intent Detection Problem: A New Approach23 Jan 2026
- 2026One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models9 Jan 2026
- 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges12 Dec 2025
- 2025MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark1 Dec 2025
- 2025Two-Dimensional Quantization for Geometry-Aware Audio Coding1 Dec 2025
- 2025Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding24 Nov 2025
- 2025Multimodal Evaluation of Russian-language Architectures19 Nov 2025
- 2025Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents11 Nov 2025
- 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Models22 Oct 2025
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue15 Oct 2025
- 2025Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialects9 Oct 2025
- 2025FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation19 Sep 2025
- 2025Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Data18 Sep 2025
- 2025MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch15 Sep 2025
- 2025SENSE models: an open source solution for multilingual and multimodal semantic-based tasks15 Sep 2025
- 2025Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs25 Aug 2025
- 2025LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model21 Aug 2025
- 2025RealTalk-CN: A Realistic Chinese Speech-Text Dialogue Benchmark With Cross-Modal Interaction Analysis6 Aug 2025
- 2025Deep Learning Approaches for Multimodal Intent Recognition: A Survey24 Jul 2025
- 2025AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation17 Jul 2025
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark5 Jun 2025
- 2025Leveraging Information Retrieval to Enhance Spoken Language Understanding Prompts in Few-Shot Learning3 Jun 2025
- 2025Pushing the Limits of Beam Search Decoding for Transducer-based ASR models30 May 2025
- 2025NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decoding28 May 2025
50 newest of 197 All 197 (JSON)
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Settingcited by 47
- 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Usecited by 7
- 2025Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agentscited by 2
- 2026Latent Preference Modeling for Multi-Session Personalized Tool Callingcited by 2
- 2025Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistantcited by 1
- 2026CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agentscited by 0
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbationscited by 0
- 2026Talk2Agent: Benchmarking Voice Interfaces for Text Agentscited by 0
- 2026Talk2Agent: Benchmarking Voice Interfaces for Text Agents30 Sep 2026
- 2026Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations4 Aug 2026
- 2026CORTIS: Text-Only Adaptation of Spoken Language Models for Task-Oriented Voice Agents19 Jun 2026
- 2026Latent Preference Modeling for Multi-Session Personalized Tool Calling20 Apr 2026
- 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Use17 Apr 2026
- 2025Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents11 Nov 2025
- 2025Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistant25 Apr 2025
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting1 May 2024
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
- 2023CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Taskscited by 18
- 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Modelscited by 4
- 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Models22 Oct 2025
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark5 Jun 2025
- 2023CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks23 Oct 2023
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
- 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
- 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
- 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2023Textually Pretrained Speech Language Modelscited by 99
- 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
- 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
- 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
- 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 77
- 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
12 most cited of 178 All 178 (JSON)
- 2026EdgeCraft: Automated Model Crafting for Edge IoT28 Sep 2026
- 2026Quieter Than the Room: Representation Drift and Task Robustness in Speech Encoders23 Sep 2026
- 2026Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs19 Sep 2026
- 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding10 Sep 2026
- 2026Topic Matching in the Wild: Benchmark and Lessons from Real-World ASR Transcripts27 Aug 2026
- 2026Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text24 Aug 2026
- 2026Aslema at NADI 2026: Data Augmentation for Intent Recognition and Slot Filling19 Aug 2026
- 2026ESF-Bench: Benchmarking Challenging Slot-Filling Scenarios for Real-World Enterprise Applications25 Jul 2026
- 2026GigaAM Multilingual: Foundation Model for Underrepresented Languages11 Jul 2026
- 2026Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?24 Jun 2026
- 2026When Do LLMs Replace Fine-Tuned NLU? A Decision Framework for Intent Detection in Production Conversational Systems19 Jun 2026
- 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects31 May 2026
12 newest of 178 All 178 (JSON)