mlsanityData to 7 Oct 2026

Self-Improvement as Coherence Optimization: A Theoretical Account

arXiv 2601.13566 ↗First posted 20 Jan 2026cs.LG cs.AI cs.CLcited by 1

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even matching golden finetuning performance. Yet why they work remains theoretically unclear. We show that they can all be understood as coherence optimization, the search for a context-to-behavior mapping that is most compressible and jointly predictable, with debate an exact instance and bootstrap and internal coherence maximization closely related to it. We prove that coherence optimization is equivalent to description-length regularization, and that among all such regularization schemes, coherence regularization with a prior derived from a pretrained model optimizes a lower bound of worst-case accuracy for semi-supervised learning. Our theory, supported by preliminary experiments, explains why feedback-free self-improvement works and predicts when it should succeed or fail.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 29 of its 60 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

29 of 60 references.

References · 29

  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  3. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  4. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2018AI safety via debatecited by 384
  7. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  8. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  9. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  10. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  11. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  12. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
17 more
  1. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  2. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 84
  3. 2023Scalable AI Safety via Doubly-Efficient Debatecited by 54
  4. 2020Is SGD a Bayesian sampler? Well, almostcited by 50
  5. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 46
  6. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 46
  7. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  8. 2025On the generalization of language models from in-context learning and finetuning: a controlled studycited by 39
  9. 2024HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematicscited by 32
  10. 2022Composing Ensembles of Pre-trained Models via Iterative Consensuscited by 28
  11. 2016Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithmcited by 15
  12. 2025Unsupervised Elicitation of Language Modelscited by 11
  13. 2025Avoiding Obfuscation with Prover-Estimator Debatecited by 11
  14. 2016Nonparametric General Reinforcement Learningcited by 10
  15. 2024Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularizationcited by 9
  16. 2025Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorycited by 4
  17. 2024Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Datacited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprisecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Coherence Mechanisms for Provable Self-Improvement
  2. 2026Coherence Maximization Improves Pluralistic Alignment
  3. 2024Self-Improvement in Language Models: The Sharpening Mechanism
  4. 2024Don't drop your samples! Coherence-aware training benefits Conditional diffusion
  5. 2026Simulated Annealing Enhances Theory-of-Mind Reasoning in Autoregressive Language Models
  6. 2026Position: It's Time to Optimize LLMs for Self-Consistency
  7. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  8. 2025Benchmarking graph construction by large language models for coherence-driven inference