# HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training

arXiv: https://arxiv.org/abs/2507.10920 · HTML: https://mlsanity.com/paper/2507.10920/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [SEED: Self-supervised Distillation For Visual Representation](https://mlsanity.com/paper/2101.04731/) (2021)
- [KOBEST: Korean Balanced Evaluation of Significant Tasks](https://mlsanity.com/paper/2204.04541/) (2022)
- [Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2105.03953/) (2021)
- [KoBALT: Korean Benchmark For Advanced Linguistic Tasks](https://mlsanity.com/paper/2505.16125/) (2025)
- [Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training](https://mlsanity.com/paper/2411.14318/) (2024)
- [RedWhale: An Adapted Korean LLM Through Efficient Continual Pretraining](https://mlsanity.com/paper/2408.11294/) (2024)
- [InstructionCP: A fast approach to transfer Large Language Models into target language](https://mlsanity.com/paper/2405.20175/) (2024)
- [Korean-to-Chinese Machine Translation using Chinese Character as Pivot Clue](https://mlsanity.com/paper/1911.11008/) (2019)

## Abstract (arXiv)

Large language models (LLMs) often show poor performance in low-resource languages like Korean, partly due to unique linguistic challenges such as homophonous Sino-Korean words that are indistinguishable in Hangul script. To address this semantic ambiguity, we propose HanjaBridge, a novel meaning-injection technique integrated into a continual pre-training (CPT) framework. Instead of deterministically mapping a word to a single Hanja (Chinese character), HanjaBridge presents the model with all possible Hanja candidates for a given homograph, encouraging the model to learn contextual disambiguation. This process is paired with token-level knowledge distillation to prevent catastrophic forgetting. Experimental results show that HanjaBridge significantly improves Korean language understanding, achieving a 21% relative improvement on the KoBALT benchmark. Notably, by reinforcing semantic alignment between Korean and Chinese through shared Hanja, we observe a strong positive cross-lingual transfer. Furthermore, these gains persist even when Hanja augmentation is omitted at inference time, ensuring practical efficiency with no additional run-time cost.