Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features
Not on the map. Why?
Abstract (arXiv)
LLMs often struggle with modern Korean poetry, producing outputs that resemble "line-broken prose." We address two coupled tasks: detecting whether a Korean poem is human- or LLM-authored, and guiding LLMs to generate poetry closer in form to human writing. We quantify the human-LLM gap along four form-level linguistic dimensions: output length (Volume), the diversity and connective use of line-final forms (Structure Variation), the irregularity of line lengths (Rhythmic Irregularity), and adherence to standard orthography (Normative Adherence). We operationalize these dimensions as five interpretable features. For detection, a logistic regression classifier over these five features attains an average AUC-ROC of 83.60 in zero-shot out-of-distribution detection across seven unseen LLMs, versus 75.84 for the strongest baseline in our comparison, KatFishNet, an absolute gain of 7.76 AUC points and a 10.23% relative improvement; one generator-specific punctuation pattern outside our taxonomy remains a boundary case. For generation, expert evaluation on GPT-5.2 prefers feature-guided poems over the unconstrained baseline, and analyses across GPT-5.2 and Gemini-3 show that targeted length, rhythm, and ending statistics move toward the human distribution. These results suggest that interpretable, language-specific features can bridge the diagnosis and guidance of LLM-generated poetry.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 29 Aug 2026.
Builds on
36 of 58 references.
References · 36
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
- 2019Release Strategies and the Social Impacts of Language Modelscited by 505
- 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
- 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
- 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
- 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
- 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
24 more
- 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
- 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
- 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 114
- 2021KLUE: Korean Language Understanding Evaluationcited by 113
- 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
- 2024Do LLMs write like humans? Variation in grammatical and rhetorical stylescited by 62
- 2018Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhymecited by 35
- 2024Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScorecited by 27
- 2022ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Modelscited by 25
- 2022Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Featurescited by 24
- 2022PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generationcited by 22
- 2024CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLMcited by 18
- 2024StyloAI: Distinguishing AI-Generated Content with Stylometric Analysiscited by 17
- 2023Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarkscited by 14
- 2025Comparing LLM-generated and human-authored news text using formal syntactic theorycited by 11
- 2020Acrostic Poem Generationcited by 10
- 2024On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writingcited by 10
- 2025Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communicationcited by 9
- 2025The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writingcited by 7
- 2023PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generationcited by 6
- 2025KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysiscited by 6
- 2025What Makes a Good Natural Language Prompt?cited by 6
- 2025Chandomitra: Towards Generating Structured Sanskrit Poetry from Natural Language Inputscited by 5
- 2025so much depends / upon / a whitespace: Why Whitespace Matters for Poets and LLMscited by 4
Built on by
Not cited yet.