mlsanityData to 7 Oct 2026

CrossAligner & Co: Zero-Shot Transfer Methods for Task-Oriented Cross-lingual Natural Language Understanding

arXiv 2203.09982 ↗First posted 18 Mar 2022cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Task-oriented personal assistants enable people to interact with a host of devices and services using natural language. One of the challenges of making neural dialogue systems available to more users is the lack of training data for all but a few languages. Zero-shot methods try to solve this issue by acquiring task knowledge in a high-resource language such as English with the aim of transferring it to the low-resource language(s). To this end, we introduce CrossAligner, the principal method of a variety of effective approaches for zero-shot cross-lingual transfer based on learning alignment from unlabelled parallel data. We present a quantitative analysis of individual methods as well as their weighted combinations, several of which exceed state-of-the-art (SOTA) scores as evaluated across nine languages, fifteen test sets and three benchmark multilingual datasets. A detailed qualitative error analysis of the best methods shows that our fine-tuned language models can zero-shot transfer the task knowledge better than anticipated.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

28 of 41 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  7. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  8. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  11. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  12. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
16 more
  1. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 156
  2. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  3. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  4. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  5. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  6. 2020Multi-Loss Weighting with Coefficient of Variationscited by 36
  7. 2019Entity Projection via Machine Translation for Cross-Lingual NERcited by 32
  8. 2020Multilingual BERT Post-Pretraining Alignmentcited by 30
  9. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  10. 2021MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learningcited by 21
  11. 2020DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionariescited by 20
  12. 2021Rethinking InfoNCE: How Many Negative Samples Do You Need?cited by 18
  13. 2020Conversation Graph: Data Augmentation, Training and Evaluation for Non-Deterministic Dialogue Managementcited by 13
  14. 2021XeroAlign: Zero-Shot Cross-lingual Transformer Alignmentcited by 11
  15. 2021Cross-Lingual Named Entity Recognition Using Parallel Corpus: A New Approach Using XLM-RoBERTa Alignmentcited by 9
  16. 2017A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Managementcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4.

7 papers.

  1. 2023Transfer-Free Data-Efficient Multilingual Slot Labelingcited by 4
  2. 2022Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understandingcited by 3
  3. 2022Bootstrapping Multilingual Semantic Parsers using Large Language Modelscited by 2
  4. 2023Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsingcited by 2
  5. 2022Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunitiescited by 1
  6. 2023Towards Zero-Shot Functional Compositionality of Language Modelscited by 1
  7. 2023LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modelingcited by 0