# Coherence Mechanisms for Provable Self-Improvement

arXiv: https://arxiv.org/abs/2511.08440 · HTML: https://mlsanity.com/paper/2511.08440/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 12 of its 78 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 12 of 12)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Self-Improvement in Language Models: The Sharpening Mechanism](https://mlsanity.com/paper/2412.01951/) (2024)
- [Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?](https://mlsanity.com/paper/2303.01255/) (2023)
- [Pre-training under infinite compute](https://mlsanity.com/paper/2509.14786/) (2025)
- [Deep Bregman Divergence for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2109.07455/) (2021)
- [A scaled Bregman theorem with applications](https://mlsanity.com/paper/1607.00360/) (2016)
- [Playing Large Games with Oracles and AI Debate](https://mlsanity.com/paper/2312.04792/) (2023)

## Built on by (most cited first, 3 of 3)

- [Distributional Alignment Games for Answer-Level Fine-Tuning](https://mlsanity.com/paper/2604.27166/) (2026)
- [Calibration without Ground Truth](https://mlsanity.com/paper/2601.19862/) (2026)
- [A Task-Centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula](https://mlsanity.com/paper/2602.10014/) (2026)

## Abstract (arXiv)

Self-improvement is a critical capability for large language models and other intelligent systems, enabling them to refine their behavior and internal consistency without external supervision. Despite its importance, prior approaches largely rely on empirical heuristics and lack formal guarantees. In this paper, we propose a principled framework for self-improvement based on the concept of coherence, which requires that a model's outputs remain consistent under task-preserving transformations of the input. We formalize this concept using projection-based mechanisms that update a baseline model to be coherent while remaining as close as possible to its original behavior. We provide rigorous theoretical guarantees that these mechanisms achieve monotonic improvement, measured by a reduction in expected Bregman divergence. Our analysis is comprehensive, covering both direct and two-step projection methods, and robustly extends these guarantees to non-realizable settings, empirical (finite-sample) distributions, and relaxed coherence constraints. Furthermore, we establish a general characterization theorem, showing that any mechanism with similar provable improvement guarantees must inherently conform to a coherence-based structure. This culminates in rigidity results under the demand for universal improvement, establishing coherence as a fundamental and, in a formal sense, necessary principle for provable self-improvement.