# Improving Korean NLP Tasks with Linguistically Informed Subword Tokenization and Sub-character Decomposition

arXiv: https://arxiv.org/abs/2311.03928 · HTML: https://mlsanity.com/paper/2311.03928/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification](https://mlsanity.com/paper/1908.11828/) (2019)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection](https://mlsanity.com/paper/2005.12503/) (2020)
- [An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks](https://mlsanity.com/paper/2010.02534/) (2020)
- [KR-BERT: A Small-Scale Korean-Specific Language Model](https://mlsanity.com/paper/2008.03979/) (2020)
- [Segmentation-free Compositional n-gram Embedding](https://mlsanity.com/paper/1809.00918/) (2018)

## Built on by (most cited first, 2 of 2)

- [Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Models](https://mlsanity.com/paper/2503.22968/) (2025)
- [Thunder-DeID: Accurate and Efficient De-identification Framework for Korean Court Judgments](https://mlsanity.com/paper/2506.15266/) (2025)

## Abstract (arXiv)

We introduce a morpheme-aware subword tokenization method that utilizes sub-character decomposition to address the challenges of applying Byte Pair Encoding (BPE) to Korean, a language characterized by its rich morphology and unique writing system. Our approach balances linguistic accuracy with computational efficiency in Pre-trained Language Models (PLMs). Our evaluations show that this technique achieves good performances overall, notably improving results in the syntactic task of NIKL-CoLA. This suggests that integrating morpheme type information can enhance language models' syntactic and semantic capabilities, indicating that adopting more linguistic insights can further improve performance beyond standard morphological analysis.