mlsanityData to 14 Oct 2026

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

arXiv 2403.18058 ↗First posted 26 Mar 2024cs.CL cs.AIcited by 45

Not on the map. Why?

Abstract (arXiv)

Remarkable progress on English instruction tuning has facilitated the efficacy and reliability of large language models (LLMs). However, there remains a noticeable gap in instruction tuning for Chinese, where the complex linguistic features pose significant challenges. Existing datasets, generally distilled from English-centric LLMs, are not well-aligned with Chinese users' interaction patterns. To bridge this gap, we introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world resources and undergoing rigorous human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks. Additionally, our findings offer several insights for designing effective Chinese instruction-tuning datasets and data-mixing strategies. Our dataset are available at https://huggingface.co/datasets/m-a-p/COIG-CQIA.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

30 of 47 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  11. 2024Yi: Open Foundation Models by 01.AIcited by 719
  12. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
18 more
  1. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  2. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  3. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  4. 2024Large Language Models: A Surveycited by 525
  5. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  6. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  7. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 481
  8. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  9. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  10. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  11. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  12. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 229
  13. 2023Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Casescited by 86
  14. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 85
  15. 2023Chinese Open Instruction Generalist: A Preliminary Releasecited by 35
  16. 2021The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoderscited by 32
  17. 2023MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Followingcited by 30
  18. 2023Dynamics of Instruction Fine-Tuning for Chinese Large Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 22; 2025: 15; 2026: 8.

45 papers.

Sort:

By area:

  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  2. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  3. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  4. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  5. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  6. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  7. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  8. 2024GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AIcited by 24
  9. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 18
  10. 2025Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Modelscited by 13
  11. 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 12
  12. 2024MMRel: Benchmarking Relation Understanding in Multi-Modal Large Language Modelscited by 10
33 more
  1. 2024FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Datacited by 10
  2. 2024Vision Language Models See What You Want but not What You Seecited by 10
  3. 2025Evaluating Text Creativity across Diverse Domains: A Dataset and Large Language Model Evaluatorcited by 10
  4. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 7
  5. 2024BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baselinecited by 6
  6. 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuningcited by 5
  7. 2024Chumor 2.0: Towards Benchmarking Chinese Humor Understandingcited by 4
  8. 2025Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilizationcited by 4
  9. 2024Chumor 1.0: A Truly Funny and Challenging Chinese Humor Understanding Dataset from Ruo Zhi Bacited by 3
  10. 2024CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMscited by 3
  11. 2024CHBench: A Chinese Dataset for Evaluating Health in Large Language Modelscited by 3
  12. 2025SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarioscited by 3
  13. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Modelscited by 3
  14. 2026Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMscited by 3
  15. 2024Small Language Model as Data Prospector for Large Language Modelcited by 2
  16. 2025Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratchcited by 2
  17. 2025Provable Secure Steganography Based on Adaptive Dynamic Samplingcited by 2
  18. 2024Research on Tibetan Tourism Viewpoints information generation system based on LLMcited by 1
  19. 2024Educational-Psychological Dialogue Robot Based on Multi-Agent Collaborationcited by 1
  20. 2025Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesiscited by 1
  21. 2024When Emotional Stimuli meet Prompt Designing: An Auto-Prompt Graphical Paradigmcited by 0
  22. 2024Learning from "Silly" Questions Improves Large Language Models, But Only Slightlycited by 0
  23. 2025Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMscited by 0
  24. 2025TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generationcited by 0
  25. 2025Detecting Reading-Induced Confusion Using EEG and Eye Trackingcited by 0
  26. 2025COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with Thought Processescited by 0
  27. 2026The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Modelscited by 0
  28. 2026GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Modelscited by 0
  29. 2026ML-Embed: Inclusive and Efficient Embeddings for a Multilingual Worldcited by 0
  30. 2026ShopX: A Foundation Model for Intent-to-Item Fulfillment in Agentic Shoppingcited by 0
  31. 2026Towards High-Level Semantic Intelligencecited by 0
  32. 2026MMAC: A Massive Multi-dimensional Benchmark for Audio Captioningcited by 0
  33. 2026Chinese-Jev: Bringing System One Model to Chinese-Language Taskscited by 0