mlsanityData to 7 Oct 2026

LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging

arXiv 2209.09900 ↗First posted 20 Sep 2022cs.CL cs.AI cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

We present LINGUIST, a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilingual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt. In a 10-shot novel intent setting for the SNIPS dataset, LINGUIST surpasses state-of-the-art approaches (Back-Translation and Example Extrapolation) by a wide margin, showing absolute improvement for the target intents of +1.9 points on IC Recall and +2.5 points on ST F1 Score. In the zero-shot cross-lingual setting of the mATIS++ dataset, LINGUIST out-performs a strong baseline of Machine Translation with Slot Alignment by +4.14 points absolute on ST F1 Score across 6 languages, while matching performance on IC. Finally, we verify our results on an internal large-scale multilingual dataset for conversational agent IC+ST and show significant improvements over a baseline which uses Back-Translation, Paraphrasing and Slot Catalog Resampling. To our knowledge, we are the first to demonstrate instruction fine-tuning of a large-scale seq2seq model to control the outputs of multilingual intent- and slot-labeled data generation.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 20 Sep 2022.

Builds on

34 of 54 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2018Hierarchical Neural Story Generationcited by 1,182
  12. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
22 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  2. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  4. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  5. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  6. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  7. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  8. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  9. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
  10. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  11. 2021Towards Zero-Label Language Learningcited by 76
  12. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 75
  13. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  14. 2021Neural Data Augmentation via Example Extrapolationcited by 54
  15. 2016Neural Paraphrase Generation with Stacked Residual LSTM Networkscited by 52
  16. 2020SeqMix: Augmenting Active Sequence Labeling via Sequence Mixupcited by 52
  17. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  18. 2019Controlled Text Generation for Data Augmentation in Intelligent Artificial Agentscited by 20
  19. 2021FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understandingcited by 17
  20. 2022Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systemscited by 15
  21. 2021Generative Conversational Networkscited by 11
  22. 2018A Re-ranker Scheme for Integrating Large Scale NLU modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 13; 2024: 13; 2025: 1; 2026: 1.

29 papers.

Sort:

By area:

  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  2. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  3. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  4. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  5. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  6. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  7. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Modelscited by 22
  8. 2022CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsingcited by 21
  9. 2024LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Textcited by 12
  10. 2024Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuningcited by 11
  11. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Datacited by 9
  12. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
17 more
  1. 2023Toward Unified Controllable Text Generation via Regular Expression Instructioncited by 7
  2. 2024Grounding Data Science Code Generation with Input-Output Specificationscited by 7
  3. 2024Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Datacited by 7
  4. 2023Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answeringcited by 6
  5. 2024PersonalityChat: Conversation Distillation for Personalized Dialog Modeling with Facts and Traitscited by 6
  6. 2023Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-trainingcited by 5
  7. 2024ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Fillercited by 5
  8. 2023KwaiYiiMath: Technical Reportcited by 4
  9. 2023Decoupling Representation and Knowledge for Few-Shot Intent Classification and Slot Fillingcited by 2
  10. 2024Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent Classificationcited by 2
  11. 2023Evaluating Inter-Bilingual Semantic Parsing for Indian Languagescited by 1
  12. 2023Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Modelscited by 1
  13. 2024Better Python Programming for all: With the focus on Maintainabilitycited by 0
  14. 2024On the Necessity of World Knowledge for Mitigating Missing Labels in Extreme Classificationcited by 0
  15. 2024CALICO: Conversational Agent Localization via Synthetic Data Generationcited by 0
  16. 2025Integrating gender inclusivity into large language models via instruction tuningcited by 0
  17. 2026Language-Aware Token Boosting: LLM Language Confusion Reduction Without Tuningcited by 0