mlsanityData to 14 Oct 2026

Enhancing Cross-lingual Transfer by Manifold Mixup

arXiv 2205.04182 ↗First posted 9 May 2022cs.CL cs.AIcited by 30

Not on the map. Why?

Abstract (arXiv)

Based on large-scale pre-trained multilingual representations, recent cross-lingual transfer methods have achieved impressive transfer performances. However, the performance of target languages still lags far behind the source language. In this paper, our analyses indicate such a performance gap is strongly associated with the cross-lingual representation discrepancy. To achieve better cross-lingual transfer performance, we propose the cross-lingual manifold mixup (X-Mixup) method, which adaptively calibrates the representation discrepancy and gives a compromised representation for target languages. Experiments on the XTREME benchmark show X-Mixup achieves 1.8% performance gains on multiple text understanding tasks, compared with strong baselines, and significantly reduces the cross-lingual representation discrepancy.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 38 of its 53 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

38 of 53 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  7. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  8. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
  9. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  12. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
26 more
  1. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  2. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 340
  3. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  4. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  5. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  6. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  7. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  8. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
  9. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  10. 2017A Survey Of Cross-lingual Word Embedding Modelscited by 143
  11. 2019Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Taskscited by 96
  12. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 85
  13. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  14. 2018Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processingcited by 63
  15. 2020ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corporacited by 63
  16. 2020On Learning Universal Representations Across Languagescited by 58
  17. 2020Explicit Alignment Objectives for Multilingual Bidirectional Encoderscited by 50
  18. 2020Inducing Language-Agnostic Multilingual Representationscited by 48
  19. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  20. 2019XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answeringcited by 35
  21. 2020VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generationcited by 34
  22. 2020FILTER: An Enhanced Fusion Method for Cross-lingual Language Understandingcited by 29
  23. 2019Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translationcited by 27
  24. 2019Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NERcited by 25
  25. 2021nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?cited by 21
  26. 2020Multilingual Transfer Learning for QA Using Translation as Data Augmentationcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 13; 2024: 10; 2025: 1; 2026: 3.

30 papers.

Sort:

  1. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  2. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 31
  3. 2022English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddingscited by 26
  4. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  5. 2023Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solutioncited by 22
  6. 2024A Survey on Mixup Augmentations and Beyondcited by 20
  7. 2023Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models?cited by 19
  8. 2023Enhancing Cross-lingual Transfer via Phonemic Transcription Integrationcited by 11
  9. 2023EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphscited by 7
  10. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  11. 2022Synergy with Translation Artifacts for Training and Inference in Multilingual Taskscited by 6
  12. 2024Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfercited by 6
18 more
  1. 2024Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languagescited by 6
  2. 2024Analysis of Multi-Source Language Training in Cross-Lingual Transfercited by 5
  3. 2024Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generationcited by 5
  4. 2023XFEVER: Exploring Fact Verification across Languagescited by 4
  5. 2024CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scriptscited by 4
  6. 2023Self-Augmentation Improves Zero-Shot Cross-Lingual Transfercited by 3
  7. 2024SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipecited by 3
  8. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
  9. 2024ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentationcited by 2
  10. 2024Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answeringcited by 2
  11. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  12. 2026Rethinking Cross-Modal Fine-Tuning: Optimizing the Interaction Between Feature Alignment and Target Fittingcited by 1
  13. 2023ID-MixGCL: Identity Mixup for Graph Contrastive Learningcited by 0
  14. 2023Test-Time Adaptation with Perturbation Consistency Learningcited by 0
  15. 2023Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classificationcited by 0
  16. 2023Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehensioncited by 0
  17. 2026XITE: Cross-lingual Interpolation for Transfer using Embeddingscited by 0
  18. 2026Harnessing Linguistic Dissimilarity for Language Generalization on Unseen Low-Resource Varietiescited by 0