mlsanityData to 7 Oct 2026

Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise

arXiv 2109.06772 ↗First posted 14 Sep 2021cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity. However, current approaches that operate in the embedding space do not take surface similarity into account. This work presents a simple yet effective strategy to imrove cross-lingual transfer between closely related varieties. We propose to augment the data of the high-resource source language with character-level noise to make the model more robust towards spelling variations. Our strategy shows consistent improvements over several languages and tasks: Zero-shot transfer of POS tagging and topic identification between language varieties from the Finnic, West and North Germanic, and Western Romance language branches. Our work provides evidence for the usefulness of simple surface-level noise in improving transfer between language varieties.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

21 of 42 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  6. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  7. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  8. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  9. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  10. 2020When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Modelscited by 131
  11. 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
  12. 2021Multi-view Subword Regularizationcited by 40
9 more
  1. 2019Improving Robustness of Machine Translation with Synthetic Noisecited by 36
  2. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  3. 2017How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?cited by 23
  4. 2021Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Trainingcited by 21
  5. 2016Robsut Wrod Reocginiton via semi-Character Recurrent Neural Networkcited by 18
  6. 2019MOROCO: The Moldavian and Romanian Dialectal Corpuscited by 15
  7. 2020Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizicited by 13
  8. 2018Neural Machine Translation of Text from Non-Native Speakerscited by 10
  9. 2019Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss Germancited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 6; 2025: 5; 2026: 3.

19 papers.

Sort:

  1. 2024Natural Language Processing for Dialects of a Language: A Surveycited by 40
  2. 2023Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languagescited by 19
  3. 2023Findings of the VarDial Evaluation Campaign 2023cited by 14
  4. 2024Evaluating Pixel Language Models on Non-Standardized Languagescited by 10
  5. 2023Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languagescited by 9
  6. 2024MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebankcited by 7
  7. 2023BERTwich: Extending BERT's Capabilities to Model Dialectal and Noisy Textcited by 6
  8. 2024Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varietiescited by 4
  9. 2024Modular Adaptation of Multilingual Encoders to Written Swiss German Dialectcited by 2
  10. 2024Modeling Orthographic Variation in Occitan's Dialectscited by 2
  11. 2025Add Noise, Tasks, or Layers? MaiNLP at the VarDial 2025 Shared Task on Norwegian Dialectal Slot and Intent Detectioncited by 2
  12. 2025Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variantscited by 2
7 more
  1. 2023CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languagescited by 1
  2. 2025Unveiling Factors for Enhanced POS Tagging: A Study of Low-Resource Medieval Romance Languagescited by 1
  3. 2025Can MLLMs Generalize to Multi-Party dialog? Exploring Multilingual Response Generation in Complex Scenarioscited by 0
  4. 2025Benchmarking NLP-supported Language Sample Analysis for Swiss Children's Speechcited by 0
  5. 2026textitVersteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Languagecited by 0
  6. 2026Small Data, Big Noise: Adversarial Training for Robust Parameter-Efficient Fine-Tuningcited by 0
  7. 2026Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustnesscited by 0