# SELF: Self-Evolution with Language Feedback

arXiv: https://arxiv.org/abs/2310.00533 · HTML: https://mlsanity.com/paper/2310.00533/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 1 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://mlsanity.com/paper/2304.06767/) (2023)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers](https://mlsanity.com/paper/2106.15772/) (2021)
- [Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning](https://mlsanity.com/paper/2302.02662/) (2023)
- [Training Language Models with Language Feedback at Scale](https://mlsanity.com/paper/2303.16755/) (2023)
- [Deductive Verification of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2306.03872/) (2023)
- [Language Model Self-improvement by Reinforcement Learning Contemplation](https://mlsanity.com/paper/2305.14483/) (2023)

## Built on by (most cited first, 24 of 24)

- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [Self-Boosting Large Language Models with Synthetic Preference Data](https://mlsanity.com/paper/2410.06961/) (2024)
- [I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm](https://mlsanity.com/paper/2408.08072/) (2024)
- [TOOLVERIFIER: Generalization to New Tools via Self-Verification](https://mlsanity.com/paper/2402.14158/) (2024)
- [Self-Improvements in Modern Agentic Systems: A Survey](https://mlsanity.com/paper/2607.13104/) (2026)
- [Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents](https://mlsanity.com/paper/2501.00430/) (2024)
- [VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding](https://mlsanity.com/paper/2403.14743/) (2024)
- [Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution](https://mlsanity.com/paper/2412.15650/) (2024)
- [Alita-G: Self-Evolving Generative Agent for Agent Generation](https://mlsanity.com/paper/2510.23601/) (2025)
- [Teaching Language Models to Self-Improve by Learning from Language Feedback](https://mlsanity.com/paper/2406.07168/) (2024)
- [The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation](https://mlsanity.com/paper/2505.18759/) (2025)
- [Aligning Instruction Tuning with Pre-training](https://mlsanity.com/paper/2501.09368/) (2025)
- [DEBATE, TRAIN, EVOLVE: Self Evolution of Language Model Reasoning](https://mlsanity.com/paper/2505.15734/) (2025)
- [Generalizable Self-Evolving Memory for Automatic Prompt Optimization](https://mlsanity.com/paper/2603.21520/) (2026)
- [Self-Improvement of Large Language Models: A Technical Overview and Future Outlook](https://mlsanity.com/paper/2603.25681/) (2026)
- [Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgment](https://mlsanity.com/paper/2405.12910/) (2024)
- [Active Inference for Self-Organizing Multi-LLM Systems: A Bayesian Thermodynamic Approach to Adaptation](https://mlsanity.com/paper/2412.10425/) (2024)
- [SERM: Self-Evolving Relevance Model with Agent-Driven Learning from Massive Query Streams](https://mlsanity.com/paper/2601.09515/) (2026)
- [MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems](https://mlsanity.com/paper/2607.28527/) (2026)
- [AgentOmnia: Scaling Agentic Models for Full-Scenario Applications](https://mlsanity.com/paper/2607.23124/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable versatility across various domains. To further advance LLMs, we propose 'SELF' (Self-Evolution with Language Feedback), a novel approach that enables LLMs to self-improve through self-reflection, akin to human learning processes. SELF initiates with a meta-skill learning process that equips the LLMs with capabilities for self-feedback and self-refinement. Subsequently, the model undergoes an iterative process of self-evolution. In each iteration, it utilizes an unlabeled dataset of instructions to generate initial responses. These responses are enhanced through self-feedback and self-refinement. The model is then fine-tuned using this enhanced data. The model undergoes progressive improvement through this iterative self-evolution process. Moreover, the SELF framework enables the model to apply self-refinement during inference, which further improves response quality. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention. The SELF framework indicates a promising direction for the autonomous evolution of LLMs, transitioning them from passive information receivers to active participants in their development.