Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation
Not on the map. Why?
Abstract (arXiv)
We study a practical question: can a small correction module fix errors in a frozen language model's outputs without degrading its base capabilities? We propose CRN v2, a lightweight logit-level correction module (~34M trainable parameters, 0.73% of the 4.65B text module) that sits atop a fully frozen Gemma 4 E2B model. The base model is never updated; only the correction module learns, via supervised fine-tuning followed by reference-free DPO on 83,400 error-correction pairs. On a 60-question domain exam (CEHRI: Certified Human-Robot Intelligence, covering facts, arithmetic, and implicit-goal reasoning), CRN v2 corrects 53.3% of base-model errors (reworded variant: 43.3%) while showing no degradation on tested capability benchmarks (MMLU/BoolQ N=200; car-wash N=8). A LoRA baseline at the matched CRN v1 budget (6.6M params, rank 19) achieves 83.3% correction but suffers 30-75% capability loss on the same benchmarks -- the correction-capability tradeoff. An ablation shows that the KL preservation term (lambda=0.1) is critical: lowering it to 0.01 degrades correction to 35.0%. A hidden-state injection variant at earlier layers (1.6M params, SFT-only) reaches 50.0%/55.8% but does not exceed logit correction; shallower injection (layer 4) drops to 30.0%/28.3%; multi-depth logit correction (~35M) reaches only 40%; and longer training (5,000 SFT + 2,000 DPO) stays at 53.3% -- none of the alternative configurations we tested exceeded the rank-128 logit result, consistent with a best-achieved result of ~53% rather than a floor. This is a study of a design principle (frozen base + logit correction + KL anchoring), not a claim of architectural novelty. All code, main-result weights, and evaluation scripts are released (deep variant as code only -- no trained deep checkpoints).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 17 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
8 of 17 references.
References · 8
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2016Learning without Forgettingcited by 2,496
- 2022Mass-Editing Memory in a Transformercited by 1,001
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
- 2026Training-Free Action Correction for VLA Model Failures via Language Feedback
- 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
- 2025High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
- 2026What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
- 2026CyberCorrect: A Cybernetic Framework for Closed-Loop Self-Correction in Large Language Models
- 2025IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models
- 2026ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement