mlsanityData to 7 Oct 2026

Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks

arXiv 2210.09588 ↗First posted 18 Oct 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Translation has played a crucial role in improving the performance on multilingual tasks: (1) to generate the target language data from the source language data for training and (2) to generate the source language data from the target language data for inference. However, prior works have not considered the use of both translations simultaneously. This paper shows that combining them can synergize the results on various multilingual sentence classification tasks. We empirically find that translation artifacts stylized by translators are the main factor of the performance gain. Based on this analysis, we adopt two training methods, SupCon and MixUp, considering translation artifacts. Furthermore, we propose a cross-lingual fine-tuning algorithm called MUSC, which uses SupCon and MixUp jointly and improves the performance. Our code is available at https://github.com/jongwooko/MUSC.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

34 of 42 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  7. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  9. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  10. 2019How multilingual is Multilingual BERT?cited by 642
  11. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  12. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
22 more
  1. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  2. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
  3. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  4. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  5. 2020The Multilingual Amazon Reviews Corpuscited by 142
  6. 2020BLEU might be Guilty but References are not Innocentcited by 109
  7. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  8. 2019Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Taskscited by 96
  9. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 95
  10. 2018Deflecting Adversarial Attacks with Pixel Deflectioncited by 71
  11. 2020MixKD: Towards Efficient Distillation of Large-scale Language Modelscited by 65
  12. 2020Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learningcited by 62
  13. 2020MixCo: Mix-up Contrastive Learning for Visual Representationcited by 62
  14. 2021No One Representation to Rule Them All: Overlapping Features of Training Methodscited by 57
  15. 2020Learning Loss for Test-Time Augmentationcited by 56
  16. 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
  17. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  18. 2020VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generationcited by 34
  19. 2022Enhancing Cross-lingual Transfer by Manifold Mixupcited by 30
  20. 2020FILTER: An Enhanced Fusion Method for Cross-lingual Language Understandingcited by 29
  21. 2017Image Classification of Melanoma, Nevus and Seborrheic Keratosis by Deep Neural Network Ensemblecited by 23
  22. 2021Mean Embeddings with Test-Time Data Augmentation for Ensembling of Representationscited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 2.

6 papers.

  1. 2023Revisiting Machine Translation for Cross-lingual Classificationcited by 40
  2. 2024Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languagescited by 9
  3. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  4. 2024Pretraining Language Models Using Translationesecited by 6
  5. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
  6. 2025The Devil Is in the Word Alignment Details: On Translation-Based Cross-Lingual Transfer for Token Classification Taskscited by 2