mlsanityData to 14 Oct 2026

Improving Dialectal Slot and Intent Detection with Auxiliary Tasks: A Multi-Dialectal Bavarian Case Study

arXiv 2501.03863 ↗First posted 7 Jan 2025cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Reliable slot and intent detection (SID) is crucial in natural language understanding for applications like digital assistants. Encoder-only transformer models fine-tuned on high-resource languages generally perform well on SID. However, they struggle with dialectal data, where no standardized form exists and training data is scarce and costly to produce. We explore zero-shot transfer learning for SID, focusing on multiple Bavarian dialects, for which we release a new dataset for the Munich dialect. We evaluate models trained on auxiliary tasks in Bavarian, and compare joint multi-task learning with intermediate-task training. We also compare three types of auxiliary tasks: token-level syntactic tasks, named entity recognition (NER), and language modelling. We find that the included auxiliary tasks have a more positive effect on slot filling than intent classification (with NER having the most positive effect), and that intermediate-task training yields more consistent performance gains. Our best-performing approach improves intent classification performance on Bavarian dialects by 5.1 and slot filling F1 by 8.4 percentage points.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

34 of 56 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  6. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  7. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  8. 2020German's Next Language Modelcited by 144
  9. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  10. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 125
  11. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 83
  12. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 70
22 more
  1. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 63
  2. 2020Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Surveycited by 51
  3. 2020Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLPcited by 50
  4. 2021From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understandingcited by 43
  5. 2020English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Toocited by 39
  6. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  7. 2016When is multitask learning effective? Semantic sequence prediction under varying data conditionscited by 21
  8. 2024What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialectscited by 21
  9. 2022Exploring the Role of Task Transferability in Large-Scale Multi-Task Learningcited by 17
  10. 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learningcited by 17
  11. 2023A Survey of Corpora for Germanic Low-Resource Languages and Dialectscited by 16
  12. 2023Findings of the VarDial Evaluation Campaign 2023cited by 14
  13. 2024AraFinNLP 2024: The First Arabic Financial NLP Shared Taskcited by 12
  14. 2020Encoding Syntactic Knowledge in Transformer Encoder for Intent Detection and Slot Fillingcited by 10
  15. 2022Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Modelscited by 10
  16. 2023ITALIC: An Italian Intent Classification Datasetcited by 10
  17. 2024Evaluating Pixel Language Models on Non-Standardized Languagescited by 10
  18. 2023Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languagescited by 9
  19. 2024Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Datacited by 8
  20. 2022Skit-S2I: An Indian Accented Speech to Intent datasetcited by 4
  21. 2023The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translationcited by 4
  22. 2024Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varietiescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialectscited by 2
  2. 2026Indirect Question Answering in English, German and Bavarian: A Challenging Task for High- and Low-Resource Languages Alikecited by 1
  3. 2026Saar-Voice: A Multi-Speaker Saarbr"ucken Dialect Speech Corpuscited by 0