# KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?

arXiv: https://arxiv.org/abs/2610.19399 · HTML: https://mlsanity.com/paper/2610.19399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [LiveBench: A Challenging, Contamination-Limited LLM Benchmark](https://mlsanity.com/paper/2406.19314/) (2024)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration](https://mlsanity.com/paper/2412.15701/) (2024)
- [Survey of Computational Approaches to Lexical Semantic Change](https://mlsanity.com/paper/1811.06278/) (2018)
- [EXAONE 3.5: Series of Large Language Models for Real-world Use Cases](https://mlsanity.com/paper/2412.04862/) (2024)
- [AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge](https://mlsanity.com/paper/2412.13670/) (2024)
- [EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes](https://mlsanity.com/paper/2507.11407/) (2025)
- [EXAONE 3.0 7.8B Instruction Tuned Language Model](https://mlsanity.com/paper/2408.03541/) (2024)
- [NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms](https://mlsanity.com/paper/2402.12261/) (2024)
- [WinoDict: Probing language models for in-context word acquisition](https://mlsanity.com/paper/2209.12153/) (2022)
- [Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication](https://mlsanity.com/paper/2503.04395/) (2025)
- [NewTerm: Benchmarking Real-Time New Terms for Large Language Models with Annual Updates](https://mlsanity.com/paper/2410.20814/) (2024)
- [Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why](https://mlsanity.com/paper/2606.00093/) (2026)

## Abstract (arXiv)

Korean neologisms arise through diverse mechanisms, a demanding test of language models beyond lexical memorization. To enable systematic evaluation, we introduce NeoTrace, an automated, rerunnable framework that collects and dates neologisms across genres. Its 4,271 entries support KoAGE, a cutoff-aware benchmark with three acquisition and two generation tasks---the latter, to our knowledge, the first to compare model coinages with attested neologisms under their original materials and community context. Across 21 models, lexical recall diverges from contextual inference: in a five-model temporal analysis, models define only 13-30% of cutoff-year words, yet can infer meaning from one usage. However, structural understanding remains limited: even correct definitions barely improve formation analysis of pre-cutoff neologisms. Given word-building materials, single models often follow human formation mechanisms, whereas multi-agent coinage favors abbreviation and depends on interaction framing. KoAGE thus separates what models know, learn from use, and coin.