# SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities

arXiv: https://arxiv.org/abs/2304.07778 · HTML: https://mlsanity.com/paper/2304.07778/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 16 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [UER: An Open-Source Toolkit for Pre-training Models](https://mlsanity.com/paper/1909.05658/) (2019)
- [GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain](https://mlsanity.com/paper/2109.02555/) (2021)
- [AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation](https://mlsanity.com/paper/2009.11473/) (2020)
- [GPT-based Generation for Classical Chinese Poetry](https://mlsanity.com/paper/1907.00151/) (2019)
- [Generating Major Types of Chinese Classical Poetry in a Uniformed Framework](https://mlsanity.com/paper/2003.11528/) (2020)
- [Restoration of Fragmentary Babylonian Texts Using Recurrent Neural Networks](https://mlsanity.com/paper/2003.01912/) (2020)
- [Time-Aware Ancient Chinese Text Translation and Inference](https://mlsanity.com/paper/2107.03179/) (2021)
- [Summarising Historical Text in Modern Languages](https://mlsanity.com/paper/2101.10759/) (2021)
- [SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation](https://mlsanity.com/paper/2110.15723/) (2021)

## Built on by (most cited first, 10 of 10)

- [GujiBERT and GujiGPT: Construction of Intelligent Information Processing Foundation Language Models for Ancient Texts](https://mlsanity.com/paper/2307.05354/) (2023)
- [WenyanGPT: A Large Language Model for Classical Chinese Tasks](https://mlsanity.com/paper/2504.20609/) (2025)
- [C3Bench: A Comprehensive Classical Chinese Understanding Benchmark for Large Language Models](https://mlsanity.com/paper/2405.17732/) (2024)
- [TongGu: Mastering Classical Chinese Understanding with Knowledge-Grounded Large Language Models](https://mlsanity.com/paper/2407.03937/) (2024)
- [Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents](https://mlsanity.com/paper/2411.04822/) (2024)
- [RiverEcho: Real-Time Interactive Digital System for Ancient Yellow River Culture](https://mlsanity.com/paper/2506.21865/) (2025)
- [InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling](https://mlsanity.com/paper/2508.15791/) (2025)
- [Exploring the Capabilities of ChatGPT in Ancient Chinese Translation and Person Name Recognition](https://mlsanity.com/paper/2312.15304/) (2023)
- [When the Knowledge Base Becomes the Gold Standard: Measuring Resource-Shared Evaluation Loops in Entity-Level Machine Translation](https://mlsanity.com/paper/2608.11843/) (2026)
- [L"ushi-1840: Building and Probing a Knowledge Cutoff for a Classical Chinese Language Model](https://mlsanity.com/paper/2610.16201/) (2026)

## Abstract (arXiv)

The rapid advance in artificial intelligence technology has facilitated the prosperity of digital humanities research. Against such backdrop, research methods need to be transformed in the intelligent processing of ancient texts, which is a crucial component of digital humanities research, so as to adapt to new development trends in the wave of AIGC. In this study, we propose a GPT model called SikuGPT based on the corpus of Siku Quanshu. The model's performance in tasks such as intralingual translation and text classification exceeds that of other GPT-type models aimed at processing ancient texts. SikuGPT's ability to process traditional Chinese ancient texts can help promote the organization of ancient information and knowledge services, as well as the international dissemination of Chinese ancient culture.