# HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea

arXiv: https://arxiv.org/abs/2210.05112 · HTML: https://mlsanity.com/paper/2210.05112/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 5 of 5)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Latin BERT: A Contextual Language Model for Classical Philology](https://mlsanity.com/paper/2009.10053/) (2020)
- [AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation](https://mlsanity.com/paper/2009.11473/) (2020)
- [Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation](https://mlsanity.com/paper/2104.05964/) (2021)

## Built on by (most cited first, 5 of 5)

- [HistRED: A Historical Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/2307.04285/) (2023)
- [T5 meets Tybalt: Author Attribution in Early Modern English Drama Using Large Language Models](https://mlsanity.com/paper/2310.18454/) (2023)
- [DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms](https://mlsanity.com/paper/2503.03644/) (2025)
- [HERITAGE: An End-to-End Web Platform for Processing Korean Historical Documents in Hanja](https://mlsanity.com/paper/2501.11951/) (2025)
- [Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents](https://mlsanity.com/paper/2411.04822/) (2024)

## Abstract (arXiv)

Historical records in Korea before the 20th century were primarily written in Hanja, an extinct language based on Chinese characters and not understood by modern Korean or Chinese speakers. Historians with expertise in this time period have been analyzing the documents, but that process is very difficult and time-consuming, and language models would significantly speed up the process. Toward building and evaluating language models for Hanja, we release the Hanja Understanding Evaluation dataset consisting of chronological attribution, topic classification, named entity recognition, and summary retrieval tasks. We also present BERT-based models continued training on the two major corpora from the 14th to the 19th centuries: the Annals of the Joseon Dynasty and Diaries of the Royal Secretariats. We compare the models with several baselines on all tasks and show there are significant improvements gained by training on the two corpora. Additionally, we run zero-shot experiments on the Daily Records of the Royal Court and Important Officials (DRRI). The DRRI dataset has not been studied much by the historians, and not at all by the NLP community.