# Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generation

arXiv: https://arxiv.org/abs/2607.09803 · HTML: https://mlsanity.com/paper/2607.09803/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 45 of its 46 references that are in the map.
- First posted on arXiv 9 Jul 2026.

## Builds on (most cited first, 30 of 45)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification](https://mlsanity.com/paper/2308.07921/) (2023)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Randomized Autoregressive Visual Generation](https://mlsanity.com/paper/2411.00776/) (2024)
- [PruneVid: Visual Token Pruning for Efficient Video Large Language Models](https://mlsanity.com/paper/2412.16117/) (2024)
- [Making Large Language Models Better Reasoners with Alignment](https://mlsanity.com/paper/2309.02144/) (2023)
- [Forgetting Transformer: Softmax Attention with a Forget Gate](https://mlsanity.com/paper/2503.02130/) (2025)
- [Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models](https://mlsanity.com/paper/2402.12563/) (2024)
- [Large Language Models have Intrinsic Self-Correction Ability](https://mlsanity.com/paper/2406.15673/) (2024)
- [SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.08989/) (2025)
- [SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction](https://mlsanity.com/paper/2410.09008/) (2024)
- [Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalization](https://mlsanity.com/paper/2406.09416/) (2024)
- [ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification](https://mlsanity.com/paper/2502.14565/) (2025)
- [Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle](https://mlsanity.com/paper/2509.16679/) (2025)
- [Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation](https://mlsanity.com/paper/2505.24787/) (2025)
- [Pard: Permutation-Invariant Autoregressive Diffusion for Graph Generation](https://mlsanity.com/paper/2402.03687/) (2024)
- [MemoryMamba: Memory-Augmented State Space Model for Defect Recognition](https://mlsanity.com/paper/2405.03673/) (2024)
- [Improving Cross-modal Alignment for Text-Guided Image Inpainting](https://mlsanity.com/paper/2301.11362/) (2023)
- [ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies](https://mlsanity.com/paper/2506.12830/) (2025)
- [Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models](https://mlsanity.com/paper/2507.02778/) (2025)
- [Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization](https://mlsanity.com/paper/2502.05605/) (2025)
- [Self-Cognition in Large Language Models: An Exploratory Study](https://mlsanity.com/paper/2407.01505/) (2024)
- [Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss](https://mlsanity.com/paper/2602.07022/) (2026)
- [Decomposing LLM Self-Correction: The Accuracy-Correction Paradox and Error Depth Hypothesis](https://mlsanity.com/paper/2601.00828/) (2025)

## Abstract (arXiv)

Large autoregressive language models exhibit a self-correction blind spot: they reliably fix identical errors when attributed to an external source yet fail to fix the same errors in their own outputs. Prior work has documented this phenomenon empirically, through controlled error injection, error-depth decompositions, RL-based verifier-corrector training, and intrinsic self-verification, but offers no formal model of why generating a token suppresses the ability to detect its error, no quantitative activation condition for correction markers, and no convergence guarantee for reinforcement-learning-based self-correction. We close these gaps with SPARC, a spectral-algebraic theory of self-correction in autoregressive generation. We define the error-propagation operator as the product of per-step attention Jacobians on the residual stream and prove that the blind spot arises if and only if the spectral radius of this operator is at least one. We derive a sharp activation threshold, given as a function of the spectral radius, that a correction marker must exceed, recovering the 89.3% blind-spot reduction observed with a simple ``Wait'' marker. We further prove that RL-based verifier-corrector training converges at a rate proportional to the squared coupling strength over the square root of the number of samples if and only if the verifier-corrector coupling matrix has spectral norm below one, and that this criterion is invariant across residual-stream autoregressive modalities, unifying text LLMs and autoregressive image and video generation. Experiments across four backbones and a visual autoregressive probe validate every theorem, with spectral predictions matching measured blind-spot rates within 3.2% RMSE.