mlsanityData to 7 Oct 2026

CINO: A Chinese Minority Pre-trained Language Model

arXiv 2202.13558 ↗First posted 28 Feb 2022cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Multilingual pre-trained language models have shown impressive performance on cross-lingual tasks. It greatly facilitates the applications of natural language processing on low-resource languages. However, there are still some languages that the current multilingual models do not perform well on. In this paper, we propose CINO (Chinese Minority Pre-trained Language Model), a multilingual pre-trained language model for Chinese minority languages. It covers Standard Chinese, Yue Chinese, and six other ethnic minority languages. To evaluate the cross-lingual ability of the multilingual model on ethnic minority languages, we collect documents from Wikipedia and news websites, and construct two text classification datasets, WCM (Wiki-Chinese-Minority) and CMNews (Chinese-Minority-News). We show that CINO notably outperforms the baselines on various classification tasks. The CINO model and the datasets are publicly available at http://cino.hfl-rc.com.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

20 of 32 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  6. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  7. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  8. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  9. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  10. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  11. 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
  12. 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 168
8 more
  1. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  2. 2020Multilingual Alignment of Contextual Word Representationscited by 113
  3. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  4. 2019Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Taskscited by 96
  5. 2020ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corporacited by 63
  6. 2020IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLPcited by 57
  7. 2020VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generationcited by 34
  8. 2020Multilingual BERT Post-Pretraining Alignmentcited by 30

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 4; 2025: 2.

11 papers.

  1. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  2. 2024Facilitating large language model Russian adaptation with Learned Embedding Propagationcited by 8
  3. 2022TiBERT: Tibetan Pre-trained Language Modelcited by 3
  4. 2024Pay Attention to the Robustness of Chinese Minority Language Models! Syllable-level Textual Adversarial Attack on Tibetan Scriptcited by 3
  5. 2023MediaGPT : A Large Language Model For Chinese Mediacited by 2
  6. 2022MiLMo:Minority Multilingual Pre-trained Language Modelcited by 1
  7. 2023PEFTT: Parameter-Efficient Fine-Tuning for low-resource Tibetan pre-trained language modelscited by 1
  8. 2024TSCheater: Generating High-Quality Tibetan Adversarial Texts via Visual Similaritycited by 1
  9. 2025Vocabulary Customization for Efficient Domain-Specific LLM Deploymentcited by 1
  10. 2024Leveraging Retrieval-Augmented Generation for Culturally Inclusive Hakka Chatbots: Design Insights and User Perceptionscited by 0
  11. 2025AttnBoost: Retail Supply Chain Sales Insights via Gradient Boosting Perspectivecited by 0