HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding
Not on the map. Why?
Abstract (arXiv)
Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose to use consistency regularization based on a hybrid data augmentation strategy. The consistency regularization enforces the predicted distributions for an example and its semantically equivalent augmentation to be consistent. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Experimental results demonstrate that our proposed method improves the performance on both intent detection and slot filling tasks. Our systemfootnoteThe code will be available at urlhttps://github.com/bozheng-hit/MMNLU-22-HIT-SCIR. ranked 1st in the MMNLU-22 competition under the full-dataset setting.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 17 references that are in the map. First posted on arXiv 5 Jan 2023.
Builds on
17 of 17 references.
References · 17
- 2017Attention Is All You Needcited by 59,932
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
- 2019Cross-lingual Language Model Pretrainingcited by 1,088
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
- 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
- 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
- 2020Adversarial Training for Large Neural Language Modelscited by 109
- 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
- 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
5 more
- 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignmentcited by 41
- 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
- 2019XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answeringcited by 35
- 2021Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-trainingcited by 22
- 2022GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understandingcited by 14
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.
2 papers.