# MiChao-HuaFen 1.0: A Specialized Pre-trained Corpus Dataset for Domain-specific Large Models

arXiv: https://arxiv.org/abs/2309.13079 · HTML: https://mlsanity.com/paper/2309.13079/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [FinGPT: Open-Source Financial Large Language Models](https://mlsanity.com/paper/2306.06031/) (2023)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task](https://mlsanity.com/paper/2304.01097/) (2023)
- [Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture](https://mlsanity.com/paper/2306.16092/) (2023)
- [Lawyer LLaMA Technical Report](https://mlsanity.com/paper/2305.15062/) (2023)
- [WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models](https://mlsanity.com/paper/2308.10755/) (2023)
- [A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3](https://mlsanity.com/paper/2302.05729/) (2023)
- [Don't Ignore Dual Logic Ability of LLMs while Privatizing: A Data-Intensive Analysis in Medical Domain](https://mlsanity.com/paper/2309.04198/) (2023)

## Built on by (most cited first, 2 of 2)

- [Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Data](https://mlsanity.com/paper/2505.05427/) (2025)
- [Data Science and Technology Towards AGI Part I: Tiered Data Management](https://mlsanity.com/paper/2602.09003/) (2026)

## Abstract (arXiv)

With the advancement of deep learning technologies, general-purpose large models such as GPT-4 have demonstrated exceptional capabilities across various domains. Nevertheless, there remains a demand for high-quality, domain-specific outputs in areas like healthcare, law, and finance. This paper first evaluates the existing large models for specialized domains and discusses their limitations. To cater to the specific needs of certain domains, we introduce the ``MiChao-HuaFen 1.0'' pre-trained corpus dataset, tailored for the news and governmental sectors. The dataset, sourced from publicly available internet data from 2022, underwent multiple rounds of cleansing and processing to ensure high quality and reliable origins, with provisions for consistent and stable updates. This dataset not only supports the pre-training of large models for Chinese vertical domains but also aids in propelling deep learning research and applications in related fields.