mlsanityData to 7 Oct 2026

Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages

arXiv 2303.13592 ↗First posted 23 Mar 2023cs.CL cs.AIcited by 50

Not on the map. Why?

Abstract (arXiv)

While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processing (NLP) research. The recent proliferation of Large Language Models (LLMs) compels one to ask: how capable are these systems in generating code-mixed data? In this paper, we explore prompting multilingual LLMs in a zero-shot manner to generate code-mixed data for seven languages in South East Asia (SEA), namely Indonesian, Malay, Chinese, Tagalog, Vietnamese, Tamil, and Singlish. We find that publicly available multilingual instruction-tuned models such as BLOOMZ and Flan-T5-XXL are incapable of producing texts with phrases or clauses from different languages. ChatGPT exhibits inconsistent capabilities in generating code-mixed texts, wherein its performance varies depending on the prompt template and language pairing. For instance, ChatGPT generates fluent and natural Singlish texts (an English-based creole spoken in Singapore), but for English-Tamil language pair, the system mostly produces grammatically incorrect or semantically meaningless utterances. Furthermore, it may erroneously introduce languages not specified in the prompt. Based on our investigation, existing multilingual LLMs exhibit a wide range of proficiency in code-mixed data generation for SEA languages. As such, we advise against using LLMs in this context without extensive human checks.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 29 of its 94 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

29 of 94 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  9. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  10. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  11. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
  12. 2020LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluationcited by 92
17 more
  1. 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
  2. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  3. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
  4. 2023A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologiescited by 59
  5. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  6. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  7. 2021ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversationcited by 43
  8. 2019Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systemscited by 42
  9. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  10. 2022The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challengescited by 35
  11. 2021From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Textcited by 29
  12. 2020Learning Fast Adaptation on Cross-Accented Speech Recognitioncited by 28
  13. 2022M2D2: A Massively Multi-domain Language Modeling Datasetcited by 26
  14. 2021Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglotscited by 23
  15. 2021On Language Models for Creolescited by 17
  16. 2021Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixingcited by 16
  17. 2023CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarizationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 20; 2025: 15; 2026: 6.

50 papers.

Sort:

By area:

  1. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  2. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  3. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  4. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  5. 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
  6. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  7. 2024Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discriminationcited by 55
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  9. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  10. 2024Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understandingcited by 34
  11. 2024Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQcited by 27
  12. 2024Code-Switching Curriculum Learning for Multilingual Transfer in LLMscited by 26
38 more
  1. 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuningcited by 21
  2. 2025Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causescited by 20
  3. 2024MedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queriescited by 19
  4. 2023How Vocabulary Sharing Facilitates Multilingualism in LLaMA?cited by 12
  5. 2025The Multilingual Divide and Its Impact on Global AI Safetycited by 10
  6. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
  7. 2025CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languagescited by 8
  8. 2024Code-Mixer Ya Nahi: Novel Approaches to Measuring Multilingual LLMs' Code-Mixing Capabilitiescited by 7
  9. 2024Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Modelscited by 6
  10. 2023InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systemscited by 5
  11. 2024Standard Language Ideology in AI-Generated Languagecited by 5
  12. 2024Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switchingcited by 5
  13. 2025A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directionscited by 5
  14. 2025It's All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMscited by 5
  15. 2023NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languagescited by 4
  16. 2024Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Sensecited by 4
  17. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  18. 2023Persona-aware Generative Model for Code-mixed Languagecited by 3
  19. 2024Prompting Towards Alleviating Code-Switched Data Scarcity in Under-Resourced Languages with GPT as a Pivotcited by 3
  20. 2024Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysiscited by 3
  21. 2023Marathi-English Code-mixed Text Generationcited by 2
  22. 2024On Creating an English-Thai Code-switched Machine Translation in Medical Domaincited by 2
  23. 2025Conditioning LLMs to Generate Code-Switched Textcited by 2
  24. 2025CS-Sum: A Benchmark for Code-Switching Dialogue Summarization and the Limits of Large Language Modelscited by 2
  25. 2025SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMscited by 2
  26. 2025Breaking Language Barriers: Equitable Performance in Multilingual Language Modelscited by 2
  27. 2025The Impact of Code-switched Synthetic Data Quality is Task Dependent: Insights from MT and ASRcited by 1
  28. 2025Rice-VL: Evaluating Vision-Language Models for Cultural Understanding Across ASEAN Countriescited by 1
  29. 2026PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialoguescited by 1
  30. 2026Code-Switching Reveals Language Anchoring in Multilingual LLMscited by 1
  31. 2024Improving Bilingual Capabilities of Language Models to Support Diverse Linguistic Practices in Educationcited by 0
  32. 2024Collaborative Participatory Research with LLM Agents in South Asia: An Empirically-Grounded Methodological Initiative and Agenda from Field Evidence in Sri Lankacited by 0
  33. 2025MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queriescited by 0
  34. 2025Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpuscited by 0
  35. 2026ChiEngMixBench: Evaluating Large Language Models on Expert-Style Chinese-English Terminology Mixingcited by 0
  36. 2026DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American Englishcited by 0
  37. 2026Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMscited by 0
  38. 2026NLP Privacy Risk Identification in Social Media (NLP-PRISM): A Surveycited by 0