mlsanityData to 7 Oct 2026

Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots

arXiv 2103.09593 ↗First posted 17 Mar 2021cs.CL cs.AI cs.CYcited by 23

Not on the map. Why?

Abstract (arXiv)

Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix when conversing with each other. Inspired by this phenomenon, we present two strong black-box adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit. The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation. The latter directly aligns the clean example with its translations before extracting phrases as perturbations. Our phrase-level attack has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. Finally, we propose an efficient adversarial training scheme that trains in the same number of steps as the original model and show that it improves model accuracy.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 48 of its 96 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

48 of 96 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  10. 2018Robustness May Be at Odds with Accuracycited by 873
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
36 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  2. 2019How multilingual is Multilingual BERT?cited by 642
  3. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  4. 2017Word Translation Without Parallel Datacited by 480
  5. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  6. 2018Generating Natural Language Adversarial Examplescited by 422
  7. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  8. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  9. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  10. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  11. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  12. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  13. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 180
  14. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  15. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
  16. 2019Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Taskscited by 96
  17. 2017Efficient Algorithms for t-distributed Stochastic Neighborhood Embeddingcited by 75
  18. 2019A Survey of Code-switched Speech and Language Processingcited by 73
  19. 2020It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbationscited by 64
  20. 2020SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweetscited by 62
  21. 2019On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Modelscited by 56
  22. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  23. 2020Generating Fluent Adversarial Examples for Natural Languagescited by 49
  24. 2016Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classificationcited by 48
  25. 2020Code-switching pre-training for neural machine translationcited by 43
  26. 2020LAReQA: Language-agnostic answer retrieval from a multilingual poolcited by 41
  27. 2019Word-level Textual Adversarial Attacking as Combinatorial Optimizationcited by 37
  28. 2020Processing South Asian Languages Written in the Latin Script: the Dakshina Datasetcited by 35
  29. 2020Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encodingcited by 22
  30. 2020Adversarial Attack and Defense of Structured Prediction Modelscited by 20
  31. 2018Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehensioncited by 19
  32. 2019From English to Code-Switching: Transfer Learning with Strong Morphological Cluescited by 18
  33. 2019Should All Cross-Lingual Embeddings Speak English?cited by 14
  34. 2020Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizicited by 13
  35. 2017Cross-language Learning with Adversarial Neural Networks: Application to Community Question Answeringcited by 6
  36. 2019Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Datacited by 3

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 3; 2023: 6; 2024: 3; 2025: 3.

23 papers.

Sort:

  1. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  2. 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 94
  3. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  4. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  5. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  6. 2023Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languagescited by 50
  7. 2022Adversarial Attacks on Image Generation With Made-Up Wordscited by 29
  8. 2021Reliability Testing for Natural Language Processing Systemscited by 25
  9. 2021Interpreting the Robustness of Neural NLP Models to Textual Perturbationscited by 17
  10. 2021Code-switched inspired losses for generic spoken dialog representationscited by 10
  11. 2025Lost in the Mix: Evaluating LLM Understanding of Code-Switched Textcited by 9
  12. 2021On the Language-specificity of Multilingual BERT and the Impact of Fine-tuningcited by 8
11 more
  1. 2022The Risks of Machine Learning Systemscited by 5
  2. 2023IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languagescited by 5
  3. 2023Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Datacited by 4
  4. 2024Extreme Miscalibration and the Illusion of Adversarial Robustnesscited by 4
  5. 2024Code-mixed Sentiment and Hate-speech Predictioncited by 4
  6. 2021AdvCodeMix: Adversarial Attack on Code-Mixed Datacited by 3
  7. 2023Persona-aware Generative Model for Code-mixed Languagecited by 3
  8. 2021Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Modelscited by 2
  9. 2025Pay Attention to Real World Perturbations! Natural Robustness Evaluation in Machine Reading Comprehensioncited by 2
  10. 2025Adapting Multilingual Models to Code-Mixed Tasks via Model Mergingcited by 1
  11. 2023SenTest: Evaluating Robustness of Sentence Encoderscited by 0