# Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean

arXiv: https://arxiv.org/abs/2403.10882 · HTML: https://mlsanity.com/paper/2403.10882/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 16 Mar 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [Universal Neural Machine Translation for Extremely Low Resource Languages](https://mlsanity.com/paper/1802.05368/) (2018)
- [Why Does ChatGPT Fall Short in Providing Truthful Answers?](https://mlsanity.com/paper/2304.10513/) (2023)
- [BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection](https://mlsanity.com/paper/2005.12503/) (2020)
- [KOBEST: Korean Balanced Evaluation of Significant Tasks](https://mlsanity.com/paper/2204.04541/) (2022)
- [Improving Pre-Trained Multilingual Models with Vocabulary Expansion](https://mlsanity.com/paper/1909.12440/) (2019)
- [KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Application](https://mlsanity.com/paper/2305.17701/) (2023)
- [A Technical Report for Polyglot-Ko: Open-Source Large-Scale Korean Language Models](https://mlsanity.com/paper/2306.02254/) (2023)

## Built on by (most cited first, 14 of 14)

- [How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?](https://mlsanity.com/paper/2406.11477/) (2024)
- [Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models?](https://mlsanity.com/paper/2406.16316/) (2024)
- [Adapting Chat Language Models Using Only Target Unlabeled Language Data](https://mlsanity.com/paper/2412.11704/) (2024)
- [KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis](https://mlsanity.com/paper/2503.00032/) (2025)
- [Overcoming Vocabulary Constraints with Pixel-level Fallback](https://mlsanity.com/paper/2504.02122/) (2025)
- [A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs](https://mlsanity.com/paper/2506.21881/) (2025)
- [Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs](https://mlsanity.com/paper/2412.14471/) (2024)
- [Polishing Every Facet of the GEM: Testing Linguistic Competence of LLMs and Humans in Korean](https://mlsanity.com/paper/2506.01237/) (2025)
- [Detecting Voice Phishing with Precision: Fine-Tuning Small Language Models](https://mlsanity.com/paper/2506.06180/) (2025)
- [Model Correlation Detection via Random Selection Probing](https://mlsanity.com/paper/2509.24171/) (2025)
- [Enhancing Korean Dependency Parsing with Morphosyntactic Features](https://mlsanity.com/paper/2503.21029/) (2025)
- [Vocabulary Customization for Efficient Domain-Specific LLM Deployment](https://mlsanity.com/paper/2509.26124/) (2025)
- [ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs](https://mlsanity.com/paper/2601.03648/) (2026)
- [LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation](https://mlsanity.com/paper/2603.22629/) (2026)

## Abstract (arXiv)

Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. Numerous big tech companies and research institutes have developed multilingual LLMs (MLLMs) to meet current demands, overlooking less-resourced languages (LRLs). This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs. First, the MLLM vocabularies of LRLs were expanded to enhance expressiveness. Second, bilingual data were used for pretraining to align the high- and less-resourced languages. Third, a high-quality small-scale instruction dataset was constructed and instruction-tuning was performed to augment the LRL. The experiments employed the Llama2 model and Korean was used as the LRL, which was quantitatively evaluated against other developed LLMs across eight tasks. Furthermore, a qualitative assessment was performed based on human evaluation and GPT4. Experimental results showed that our proposed Bllossom model exhibited superior performance in qualitative analyses compared to previously proposed Korean monolingual models.