# AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation

arXiv: https://arxiv.org/abs/2009.11473 · HTML: https://mlsanity.com/paper/2009.11473/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 15 of 15)

- [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://mlsanity.com/paper/2102.10684/) (2021)
- [LLM-based multi-agent poetry generation in non-cooperative environments](https://mlsanity.com/paper/2409.03659/) (2024)
- [SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities](https://mlsanity.com/paper/2304.07778/) (2023)
- [Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling](https://mlsanity.com/paper/2307.08074/) (2023)
- [Can Large Language Model Comprehend Ancient Chinese? A Preliminary Test on ACLUE](https://mlsanity.com/paper/2310.09550/) (2023)
- [HistRED: A Historical Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/2307.04285/) (2023)
- [Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation](https://mlsanity.com/paper/2308.00240/) (2023)
- [WenyanGPT: A Large Language Model for Classical Chinese Tasks](https://mlsanity.com/paper/2504.20609/) (2025)
- [HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea](https://mlsanity.com/paper/2210.05112/) (2022)
- [Surveying the Dead Minds: Historical-Psychological Text Analysis with Contextualized Construct Representation (CCR) for Classical Chinese](https://mlsanity.com/paper/2403.00509/) (2024)
- [TransCouplet:Transformer based Chinese Couplet Generation](https://mlsanity.com/paper/2112.01707/) (2021)
- [Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents](https://mlsanity.com/paper/2411.04822/) (2024)
- [InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling](https://mlsanity.com/paper/2508.15791/) (2025)
- [Exploring the Capabilities of ChatGPT in Ancient Chinese Translation and Person Name Recognition](https://mlsanity.com/paper/2312.15304/) (2023)
- [SAGE: From Direct Answering to Evidence-Grounded Inference for Chinese Ancient Document Understanding](https://mlsanity.com/paper/2608.24011/) (2026)

## Abstract (arXiv)

Ancient Chinese is the essence of Chinese culture. There are several natural language processing tasks of ancient Chinese domain, such as ancient-modern Chinese translation, poem generation, and couplet generation. Previous studies usually use the supervised models which deeply rely on parallel data. However, it is difficult to obtain large-scale parallel data of ancient Chinese. In order to make full use of the more easily available monolingual ancient Chinese corpora, we release AnchiBERT, a pre-trained language model based on the architecture of BERT, which is trained on large-scale ancient Chinese corpora. We evaluate AnchiBERT on both language understanding and generation tasks, including poem classification, ancient-modern Chinese translation, poem generation, and couplet generation. The experimental results show that AnchiBERT outperforms BERT as well as the non-pretrained models and achieves state-of-the-art results in all cases.