# Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detection

arXiv: https://arxiv.org/abs/2501.03870 · HTML: https://mlsanity.com/paper/2501.03870/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 7 Jan 2025.

## Builds on (most cited first, 30 of 30)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://mlsanity.com/paper/2008.09335/) (2020)
- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://mlsanity.com/paper/2004.14353/) (2020)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://mlsanity.com/paper/2005.14672/) (2020)
- [From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding](https://mlsanity.com/paper/2105.07316/) (2021)
- [NorBench -- A Benchmark for Norwegian Language Models](https://mlsanity.com/paper/2305.03880/) (2023)
- [Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise](https://mlsanity.com/paper/2109.06772/) (2021)
- [Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages](https://mlsanity.com/paper/2304.10158/) (2023)
- [NorNE: Annotating Named Entities for Norwegian](https://mlsanity.com/paper/1911.12146/) (2019)
- [Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning](https://mlsanity.com/paper/2204.11117/) (2022)
- [Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese](https://mlsanity.com/paper/2304.08823/) (2023)
- [Findings of the VarDial Evaluation Campaign 2023](https://mlsanity.com/paper/2305.20080/) (2023)
- [AraFinNLP 2024: The First Arabic Financial NLP Shared Task](https://mlsanity.com/paper/2407.09818/) (2024)
- [Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models](https://mlsanity.com/paper/2210.13029/) (2022)
- [Evaluating Pixel Language Models on Non-Standardized Languages](https://mlsanity.com/paper/2412.09084/) (2024)
- [Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages](https://mlsanity.com/paper/2303.17683/) (2023)
- [BERTwich: Extending BERT's Capabilities to Model Dialectal and Noisy Text](https://mlsanity.com/paper/2311.00116/) (2023)
- [Zero-Shot Slot and Intent Detection in Low-Resource Languages](https://mlsanity.com/paper/2304.13292/) (2023)
- [Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties](https://mlsanity.com/paper/2402.02078/) (2024)

## Built on by (most cited first, 2 of 2)

- [Standard-to-Dialect Transfer Trends Differ across Text and Speech: A Case Study on Intent and Topic Classification in German Dialects](https://mlsanity.com/paper/2510.07890/) (2025)
- [Harnessing Linguistic Dissimilarity for Language Generalization on Unseen Low-Resource Varieties](https://mlsanity.com/paper/2605.04500/) (2026)

## Abstract (arXiv)

Slot and intent detection (SID) is a classic natural language understanding task. Despite this, research has only more recently begun focusing on SID for dialectal and colloquial varieties. Many approaches for low-resource scenarios have not yet been applied to dialectal SID data, or compared to each other on the same datasets. We participate in the VarDial 2025 shared task on slot and intent detection in Norwegian varieties, and compare multiple set-ups: varying the training data (English, Norwegian, or dialectal Norwegian), injecting character-level noise, training on auxiliary tasks, and applying Layer Swapping, a technique in which layers of models fine-tuned on different datasets are assembled into a model. We find noise injection to be beneficial while the effects of auxiliary tasks are mixed. Though some experimentation was required to successfully assemble a model from layers, it worked surprisingly well; a combination of models trained on English and small amounts of dialectal data produced the most robust slot predictions. Our best models achieve 97.6% intent accuracy and 85.6% slot F1 in the shared task.