# InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling

arXiv: https://arxiv.org/abs/2508.15791 · HTML: https://mlsanity.com/paper/2508.15791/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 12 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Deciphering Oracle Bone Language with Diffusion Models](https://mlsanity.com/paper/2406.00684/) (2024)
- [CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark](https://mlsanity.com/paper/2112.13610/) (2021)
- [CharFormer: A Glyph Fusion based Attentive Framework for High-precision Character Image Denoising](https://mlsanity.com/paper/2207.07798/) (2022)
- [AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation](https://mlsanity.com/paper/2009.11473/) (2020)
- [CCPM: A Chinese Classical Poetry Matching Dataset](https://mlsanity.com/paper/2106.01979/) (2021)
- [RZCR: Zero-shot Character Recognition via Radical-based Reasoning](https://mlsanity.com/paper/2207.05842/) (2022)
- [RCRN: Real-world Character Image Restoration Network via Skeleton Extraction](https://mlsanity.com/paper/2207.07795/) (2022)
- [SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities](https://mlsanity.com/paper/2304.07778/) (2023)
- [Toward Zero-shot Character Recognition: A Gold Standard Dataset with Radical-level Annotations](https://mlsanity.com/paper/2308.00655/) (2023)
- [GujiBERT and GujiGPT: Construction of Intelligent Information Processing Foundation Language Models for Ancient Texts](https://mlsanity.com/paper/2307.05354/) (2023)
- [Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation](https://mlsanity.com/paper/2308.00240/) (2023)
- [CHisIEC: An Information Extraction Corpus for Ancient Chinese History](https://mlsanity.com/paper/2403.15088/) (2024)
- [Corpus of Chinese Dynastic Histories: Gender Analysis over Two Millennia](https://mlsanity.com/paper/2005.08793/) (2020)
- [The Uncertainty-based Retrieval Framework for Ancient Chinese CWS and POS](https://mlsanity.com/paper/2310.08496/) (2023)
- [Ancient Chinese Word Segmentation and Part-of-Speech Tagging Using Distant Supervision](https://mlsanity.com/paper/2303.01912/) (2023)

## Built on by (most cited first, 1 of 1)

- [Towards Computational Chinese Paleography](https://mlsanity.com/paper/2601.06753/) (2026)

## Abstract (arXiv)

Constructing historical language models (LMs) plays a crucial role in aiding archaeological provenance studies and understanding ancient cultures. However, existing resources present major challenges for training effective LMs on historical texts. First, the scarcity of historical language samples renders unsupervised learning approaches based on large text corpora highly inefficient, hindering effective pre-training. Moreover, due to the considerable temporal gap and complex evolution of ancient scripts, the absence of comprehensive character encoding schemes limits the digitization and computational processing of ancient texts, particularly in early Chinese writing. To address these challenges, we introduce InteChar, a unified and extensible character list that integrates unencoded oracle bone characters with traditional and modern Chinese. InteChar enables consistent digitization and representation of historical texts, providing a foundation for robust modeling of ancient scripts. To evaluate the effectiveness of InteChar, we construct the Oracle Corpus Set (OracleCS), an ancient Chinese corpus that combines expert-annotated samples with LLM-assisted data augmentation, centered on Chinese oracle bone inscriptions. Extensive experiments show that models trained with InteChar on OracleCS achieve substantial improvements across various historical language understanding tasks, confirming the effectiveness of our approach and establishing a solid foundation for future research in ancient Chinese NLP.