How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?
Not on the map. Why?
Abstract (arXiv)
Fine-tuned Large Language Models (LLMs) dominate in Ukrainian grammatical error correction (GEC), while API-accessed LLMs remain nearly untested on minimal-edit benchmarks. We evaluate 11 commercial LLMs from four providers and one open-source Ukrainian model on the UNLP 2023 GEC-only benchmark, comparing zero-shot, few-shot, minimal-edits, and LLM-assisted prompt optimization strategies. Our best configuration (Gemini 3.1-Pro) reaches F0.5=69.22, closing over 90% of the gap to fine-tuned SOTA (F0.5=73.14). For zero-shot prompts, only Claude models benefit from Ukrainian instructions. However, the best overall results for all models use Ukrainian minimal-edits prompts, whose language-specific rules require Ukrainian to express precisely. LLM-assisted prompt optimization on top of minimal-edits + few-shot achieves the highest score. Detailed minimal-edits instructions yield the largest gains for punctuation and case errors but cause the model to abandon several low-frequency categories. Delving into error analysis, we identify five recurring overcorrection patterns tied to Ukrainian-specific linguistic phenomena. Code, prompts, and outputs are publicly available.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 9 of its 30 references that are in the map. First posted on arXiv 8 Jun 2026.
Builds on
9 of 30 references.
References · 9
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
- 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
- 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 96
- 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 61
- 2023Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methodscited by 37
- 2024GPT-3.5 for Grammatical Error Correctioncited by 18
- 2025Adapting LLMs for Minimal-edit Grammatical Error Correctioncited by 5
- 2024Spivavtor: An Instruction Tuned Ukrainian Text Editing Modelcited by 4
- 2025Introducing OmniGEC: A Silver Multilingual Dataset for Grammatical Error Correctioncited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.