Do Chinese models speak Chinese languages?
Not on the map. Why?
Abstract (arXiv)
The release of top-performing open-weight LLMs has cemented China's role as a leading force in AI development. Do these models support languages spoken in China? Or do they support the same languages as models developed in the United States or in Europe? Comparing multilingual capabilities is important for two reasons. First, language ability provides insights into pre-training data curation, and thus into resource allocation and development priorities. Second, Chinese model developers need to navigate the tension between serving a linguistically diverse population domestically, and optimizing for globally visible benchmarks that are predominantly English. We investigate Chinese model developers' priorities through a comparative study of Chinese-developed and Western-developed open-weight LLMs, on 21 language variants including Asian regional, Chinese, and European languages. Our experiments on Information Parity and reading comprehension show Chinese models' performance across these languages correlates strongly (r=0.93) with their Western counterparts, with the sole exception being better Mandarin. Chinese-developed models are good at French and German, but they sometimes cannot identify languages spoken by Chinese minorities such as Kazakh and Uyghur. Overall, all open-weight LLMs we study have a similar multilingual performance profile, despite the diverse linguistic and cultural contexts the model developers operated within. We interpret the homogenization as consistent with the influence of global benchmarking practices and shared training resources. Rather than treating current language support as inevitable, our results highlight multilingual development as a space of prioritization and trade-offs, with implications for model developers, policymakers, and users.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 30 of its 71 references that are in the map. First posted on arXiv 31 Mar 2025.
Builds on
30 of 71 references.
References · 30
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2023Mistral 7Bcited by 3,164
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2024Yi: Open Foundation Models by 01.AIcited by 719
- 2023Baichuan 2: Open Large-scale Language Modelscited by 670
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
- 2024InternLM2 Technical Reportcited by 397
- 20242 OLMo 2 Furiouscited by 340
- 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
18 more
- 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
- 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
- 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
- 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
- 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
- 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
- 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
- 2022Measuring the Carbon Intensity of AI in Cloud Instancescited by 111
- 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 102
- 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 60
- 2024Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AIcited by 35
- 2023MasakhaNEWS: News Topic Classification for African languagescited by 31
- 2024A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedianscited by 29
- 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
- 2023Algorithms as Social-Ecological-Technological Systems: an Environmental Justice Lens on Algorithmic Auditscited by 12
- 2023Decolonial AI Alignment: Openness, Vi'sedsa-Dharma, and Including Excluded Knowledgescited by 10
- 2023MC2: Towards Transparent and Culturally-Aware NLP for Minority Languages in Chinacited by 4
- 2024You Still See Me: How Data Protection Supports the Architecture of AI Surveillancecited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.