mlsanityData to 7 Oct 2026

ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler

arXiv 2403.17536 ↗First posted 26 Mar 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

State-of-the-art intent classification (IC) and slot filling (SF) methods often rely on data-intensive deep learning models, limiting their practicality for industry applications. Large language models on the other hand, particularly instruction-tuned models (Instruct-LLMs), exhibit remarkable zero-shot performance across various natural language tasks. This study evaluates Instruct-LLMs on popular benchmark datasets for IC and SF, emphasizing their capacity to learn from fewer examples. We introduce ILLUMINER, an approach framing IC and SF as language generation tasks for Instruct-LLMs, with a more efficient SF-prompting method compared to prior work. A comprehensive comparison with multiple baselines shows that our approach, using the FLAN-T5 11B model, outperforms the state-of-the-art joint IC+SF method and in-context learning with GPT3.5 (175B), particularly in slot filling by 11.1--32.2 percentage points. Additionally, our in-depth ablation study demonstrates that parameter-efficient fine-tuning requires less than 6% of training data to yield comparable performance with traditional full-weight fine-tuning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

39 of 53 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
27 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  3. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  4. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  5. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  6. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  7. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  8. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  9. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  10. 2021Template-Based Named Entity Recognition Using BARTcited by 153
  11. 2021Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue Systemcited by 139
  12. 2022Efficient Few-Shot Learning Without Promptscited by 135
  13. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  14. 2021Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuningcited by 46
  15. 2020Learning to Classify Intents and Slot Labels Given a Handful of Examplescited by 31
  16. 2022LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Taggingcited by 29
  17. 2023Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Informationcited by 24
  18. 2021DS-TOD: Efficient Domain Specialization for Task Oriented Dialogcited by 21
  19. 2023Exploring Zero and Few-shot Techniques for Intent Classificationcited by 21
  20. 2022Improved and Efficient Conversational Slot Labeling through Question Answeringcited by 10
  21. 2022Bi-directional Joint Neural Networks for Intent Classification and Slot Fillingcited by 8
  22. 2023Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Promptingcited by 8
  23. 2019Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systemscited by 7
  24. 2021Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Fillingcited by 6
  25. 2022Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Taggingcited by 5
  26. 2023Zero-Shot Slot and Intent Detection in Low-Resource Languagescited by 4
  27. 2023Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Modelcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2025Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routingcited by 1
  2. 2024"My life is miserable, have to sign 500 autographs everyday": Exposing Humblebragging, the Brags in Disguisecited by 0
  3. 2025Leveraging Information Retrieval to Enhance Spoken Language Understanding Prompts in Few-Shot Learningcited by 0
  4. 2025A Computational Approach to Modeling Conversational Systems: Analyzing Large-Scale Quasi-Patterned Dialogue Flowscited by 0
  5. 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challengescited by 0