mlsanityData to 7 Oct 2026

Self-rewarding alignment and self-training

189 papers3 this weekLow confidence in this grouping: the papers may belong to more than one topic, or the group may changeLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About

Keywords: language, llms, preference, self-improvement, alignment

Key facts. Self-rewarding alignment and self-training: 189 papers on arXiv since 2023 (data through 7 Oct 2026). 39 of its 189 mapped papers (21%) cite “Self-Rewarding Language Models” (2024, arXiv 2401.10020). 32 of its 189 mapped papers (17%) cite “Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models” (2024, arXiv 2401.01335). 15 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

15 new papers, 17 Sep – 7 Oct; 3 this week.

  1. This week · 3
  2. Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models also fits: Synthetic reasoning data and bootstrapping
  3. The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules also fits: Self-play RL for reasoning
  4. ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
  5. Earlier · 12
  6. When Is Enough Enough in Self-Evolving LLM Systems?
  7. PsyEvo: A Personalized Counseling Agent That Self-Evolves at Test Time also fits: Medical synthetic data and agents
  8. Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
  9. Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness also fits: Self-play RL for reasoning
  10. Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving
  11. PADMÉ: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators
  12. Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training also fits: Synthetic reasoning data and bootstrapping
All 15
  1. COEVO: Co-Evolving Context and Parameters for Recursive Self-Improvement also fits: Self-evolving LLM agents
  2. SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought also fits: Self-play RL for reasoning
  3. StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skills also fits: Code and instruction data synthesis
  4. Strategically Diverse Sampling for Self-Training
  5. EvoAudio: Recursive Self-Improvement for Audio Understanding

Foundations

Most cited by papers here

  1. Self-Rewarding Language Models2024cited by 39 here
  2. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models2024cited by 32 here
  3. Reinforced Self-Training (ReST) for Language Modeling2023cited by 21 here
  4. Self-Play Preference Optimization for Language Model Alignment2024cited by 19 here
  5. Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge2024cited by 14 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 25 here
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 21 here
  3. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 16 here
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 32 here
  5. 2024Self-Rewarding Language Modelscited by 39 here
  6. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 10 here
  7. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 19 here
  8. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 14 here
  9. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 14 here
  10. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Modelscited by 7 here
14 more it builds on, not in this chain
  1. 2022Large Language Models Can Self-Improvecited by 34 here
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 33 here
  3. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 6 here
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 8 here
  5. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 10 here
  6. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 6 here
  7. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 6 here
  8. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 10 here
  9. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 9 here
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 13 here
  11. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 10 here
  12. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 7 here
  13. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 10 here
  14. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 8 here

The 24 most cited of 231.

Browse

30 of 189, ranked 6–35

  1. 6Self-Improvement in Language Models: The Sharpening Mechanism202414
  2. 7Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters202413
  3. 8Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences202410
  4. 9Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution202310
  5. 10Recursive Introspection: Teaching Language Model Agents How to Self-Improve20249
  6. 11Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models20247
  7. 12Language Model Self-improvement by Reinforcement Learning Contemplation20236
  8. 13Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing20245
  9. 14Progress or Regress? Self-Improvement Reversal in Post-training20245
  10. 15CREAM: Consistency Regularized Self-Rewarding Language Models20245
All 30
  1. 16On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes20235
  2. 17Self-Improving LLM Agents at Test-Time20253
  3. 18Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation20243
  4. 19SAIL: Self-Improving Efficient Online Alignment of Large Language Models20243
  5. 20Importance Weighting Can Help Large Language Models Self-Improve20243
  6. 21A Survey on LLM Inference-Time Self-Improvement20243
  7. 22BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing20233
  8. 23Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation20233
  9. 24Teaching Language Models to Self-Improve through Interactive Demonstrations20233
  10. 25Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models20252
  11. 26RSPO: Regularized Self-Play Alignment of Large Language Models20252
  12. 27Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs20252
  13. 28Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap20252
  14. 29SPACE: Noise Contrastive Estimation Stabilizes Self-Play Fine-Tuning for Large Language Models20252
  15. 30Investigating Regularization of Self-Play Language Models20242
  16. 31SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses20242
  17. 32Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models20242
  18. 33Self-Evolved Reward Learning for LLMs20242
  19. 34Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling20242
  20. 35Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMs20261