mlsanityData to 14 Oct 2026

KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?

arXiv 2610.19399 ↗First posted 14 Oct 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Korean neologisms arise through diverse mechanisms, a demanding test of language models beyond lexical memorization. To enable systematic evaluation, we introduce NeoTrace, an automated, rerunnable framework that collects and dates neologisms across genres. Its 4,271 entries support KoAGE, a cutoff-aware benchmark with three acquisition and two generation tasks---the latter, to our knowledge, the first to compare model coinages with attested neologisms under their original materials and community context. Across 21 models, lexical recall diverges from contextual inference: in a five-model temporal analysis, models define only 13-30% of cutoff-year words, yet can infer meaning from one usage. However, structural understanding remains limited: even correct definitions barely improve formation analysis of pre-cutoff neologisms. Given word-building materials, single models often follow human formation mechanisms, whereas multi-agent coinage favors abbreviation and depends on interaction framing. KoAGE thus separates what models know, learn from use, and coin.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

21 of 45 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  8. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  9. 2024Emergent social conventions and collective bias in LLM populationscited by 123
  10. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 108
  11. 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaborationcited by 65
  12. 2018Survey of Computational Approaches to Lexical Semantic Changecited by 62
9 more
  1. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 53
  2. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 44
  3. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 32
  4. 2024EXAONE 3.0 7.8B Instruction Tuned Language Modelcited by 21
  5. 2024NEO-BENCH: Evaluating Robustness of Large Language Models with Neologismscited by 20
  6. 2022WinoDict: Probing language models for in-context word acquisitioncited by 15
  7. 2025Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communicationcited by 10
  8. 2024NewTerm: Benchmarking Real-Time New Terms for Large Language Models with Annual Updatescited by 8
  9. 2026Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Whycited by 6

Built on by

Not cited yet.