mlsanityData to 7 Oct 2026

Are Multilingual Models Effective in Code-Switching?

arXiv 2103.13309 ↗First posted 24 Mar 2021cs.CL cs.LGcited by 56

Not on the map. Why?

Abstract (arXiv)

Multilingual language models have shown decent performance in multilingual and cross-lingual natural language understanding tasks. However, the power of these multilingual models in code-switching tasks has not been fully explored. In this paper, we study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting by considering the inference speed, performance, and number of parameters to measure their practicality. We conduct experiments in three language pairs on named entity recognition and part-of-speech tagging and compare them with existing methods, such as using bilingual embeddings and multilingual meta-embeddings. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching, while using meta-embeddings achieves similar results with significantly fewer parameters.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

34 of 51 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  5. 2019How multilingual is Multilingual BERT?cited by 642
  6. 2017Word Translation Without Parallel Datacited by 480
  7. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  8. 2018Learning Word Vectors for 157 Languagescited by 279
  9. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  10. 2017Advances in Pre-Training Distributed Word Representationscited by 207
  11. 2020IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understandingcited by 108
  12. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
22 more
  1. 2020LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluationcited by 92
  2. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 69
  3. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  4. 2017BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languagescited by 52
  5. 2019Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systemscited by 42
  6. 2020XPersona: Evaluating Multilingual Personalized Chatbotcited by 36
  7. 2019Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Taskcited by 24
  8. 2020Meta-Transfer Learning for Code-Switched Speech Recognitioncited by 22
  9. 2014Multilingual Models for Compositional Distributed Semanticscited by 18
  10. 2019A Multi-task Approach for Named Entity Recognition in Social Media Datacited by 18
  11. 2018Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddingscited by 15
  12. 2018Dynamic Meta-Embeddings for Improved Sentence Representationscited by 15
  13. 2018Code-Switching Language Modeling using Syntax-Aware Multi-Task Learningcited by 15
  14. 2019A Deep Generative Model for Code-Switched Textcited by 13
  15. 2018Learn to Code-Switch: Data Augmentation using Copy Mechanism on Language Modelingcited by 9
  16. 2017Think Globally, Embed Locally --- Locally Linear Meta-embedding of Wordscited by 8
  17. 2019Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognitioncited by 7
  18. 2017Linear Ensembles of Word Embedding Modelscited by 6
  19. 2017Crowdsourcing Universal Part-Of-Speech Tags for Code-Switchingcited by 4
  20. 2018Bilingual Character Representation for Efficiently Addressing Out-of-Vocabulary Words in Code-Switching Named Entity Recognitioncited by 4
  21. 2019CAiRE_HKUST at SemEval-2019 Task 3: Hierarchical Attention for Dialogue Emotion Classificationcited by 3
  22. 2021CMSAOne@Dravidian-CodeMix-FIRE2020: A Meta Embedding and Transformer model for Code-Mixed Sentiment Analysis on Social Media Textcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 9; 2024: 11; 2025: 8; 2026: 5.

56 papers.

Sort:

By area:

  1. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  2. 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 94
  3. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  4. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  5. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  6. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  7. 2023Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languagescited by 50
  8. 2021ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversationcited by 43
  9. 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
  10. 2021Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translationcited by 29
  11. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
  12. 2021RoBERTuito: a pre-trained language model for social media text in Spanishcited by 22
38 more
  1. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  2. 2024MINERS: Multilingual Language Models as Semantic Retrieverscited by 11
  3. 2021Code-switched inspired losses for generic spoken dialog representationscited by 10
  4. 2023Yes but.. Can ChatGPT Identify Entities in Historical Documents?cited by 10
  5. 2021CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixingcited by 8
  6. 2023CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarizationcited by 8
  7. 2025CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languagescited by 8
  8. 2021Quality Evaluation of the Low-Resource Synthetically Generated Code-Mixed Hinglish Textcited by 7
  9. 2022Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switchingcited by 6
  10. 2024From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representationcited by 6
  11. 2024Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Modelscited by 6
  12. 2021Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximationcited by 5
  13. 2021Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switchingcited by 5
  14. 2022SNP2Vec: Scalable Self-Supervised Pre-Training for Genome-Wide Association Studycited by 5
  15. 2022Every picture tells a story: Image-grounded controllable stylistic story generationcited by 5
  16. 2023IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languagescited by 5
  17. 2024m3P: Towards Multimodal Multilingual Translation with Multimodal Promptcited by 5
  18. 2024From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languagescited by 5
  19. 2025A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directionscited by 5
  20. 2021Multilingual Transfer Learning for Code-Switched Language and Speech Neural Modelingcited by 4
  21. 2021The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understandingcited by 4
  22. 2023Towards Real-World Streaming Speech Translation for Code-Switched Speechcited by 4
  23. 2023Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attentioncited by 4
  24. 2024Code-mixed Sentiment and Hate-speech Predictioncited by 4
  25. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  26. 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Paircited by 3
  27. 2021Call Larisa Ivanovna: Code-Switching Fools Multilingual NLU Modelscited by 2
  28. 2024Stanceosaurus 2.0: Classifying Stance Towards Russian and Spanish Misinformationcited by 2
  29. 2024Grammatical Error Correction for Code-Switched Sentences by Learners of Englishcited by 2
  30. 2025Conditioning LLMs to Generate Code-Switched Textcited by 2
  31. 2025LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriamacited by 2
  32. 2022How Effective is Incongruity? Implications for Code-mix Sarcasm Detectioncited by 1
  33. 2022DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialoguecited by 1
  34. 2022Automatic Identification of Motivation for Code-Switching in Speech Transcriptscited by 1
  35. 2024MaCmS: Magahi Code-mixed Dataset for Sentiment Analysiscited by 1
  36. 2025Multilingual Information Retrieval with a Monolingual Knowledge Basecited by 1
  37. 2025Adapting Multilingual Models to Code-Mixed Tasks via Model Mergingcited by 1
  38. 2026PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialoguescited by 1

50 most cited of 56 All 56 (JSON)