# Self-Adapting Language Models

arXiv: https://arxiv.org/abs/2506.10943 · HTML: https://mlsanity.com/paper/2506.10943/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 73 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 30 of 53)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://mlsanity.com/paper/2312.06585/) (2023)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Modifying Memories in Transformer Models](https://mlsanity.com/paper/2012.00363/) (2020)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [Will we run out of data? Limits of LLM scaling based on human-generated data](https://mlsanity.com/paper/2211.04325/) (2022)
- [Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling](https://mlsanity.com/paper/2401.16380/) (2024)

## Built on by (most cited first, 30 of 84)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [RL's Razor: Why Online Reinforcement Learning Forgets Less](https://mlsanity.com/paper/2509.04259/) (2025)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Self-Questioning Language Models](https://mlsanity.com/paper/2508.03682/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [Language Self-Play For Data-Free Training](https://mlsanity.com/paper/2509.07414/) (2025)
- [Self-Improving LLM Agents at Test-Time](https://mlsanity.com/paper/2510.07841/) (2025)
- [SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass](https://mlsanity.com/paper/2602.06358/) (2026)
- [Rethinking On-Policy Distillation of Large Language Models II: One Training Example](https://mlsanity.com/paper/2609.04172/) (2026)
- [Structured Scaling of AI Discovery Across Diverse Scientific Domains](https://mlsanity.com/paper/2604.19341/) (2026)
- [LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls](https://mlsanity.com/paper/2511.09148/) (2025)
- [Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability](https://mlsanity.com/paper/2601.18778/) (2026)
- [Understanding LoRA as Knowledge Memory: An Empirical Analysis](https://mlsanity.com/paper/2603.01097/) (2026)
- [SEAL: Synergistic Co-Evolution of Agents and Learning Environments](https://mlsanity.com/paper/2605.24426/) (2026)
- [Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning](https://mlsanity.com/paper/2510.04786/) (2025)
- [ProbeLLM: Automating Principled Diagnosis of LLM Failures](https://mlsanity.com/paper/2602.12966/) (2026)
- [Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses](https://mlsanity.com/paper/2608.24876/) (2026)
- [ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement](https://mlsanity.com/paper/2609.14857/) (2026)
- [iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models](https://mlsanity.com/paper/2601.05877/) (2026)
- [ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents](https://mlsanity.com/paper/2609.17523/) (2026)
- [ALAS: Autonomous Learning Agent for Self-Updating Language Models](https://mlsanity.com/paper/2508.15805/) (2025)
- [Synthetic bootstrapped pretraining](https://mlsanity.com/paper/2509.15248/) (2025)
- [Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2605.14892/) (2026)
- [Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models](https://mlsanity.com/paper/2509.24510/) (2025)
- [AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecycle](https://mlsanity.com/paper/2605.31468/) (2026)
- [Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection](https://mlsanity.com/paper/2608.20169/) (2026)
- [Aspire: Can Models Self-Evolve from Vague Goals?](https://mlsanity.com/paper/2608.31111/) (2026)
- [No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers](https://mlsanity.com/paper/2512.08889/) (2025)
- [Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation](https://mlsanity.com/paper/2601.11258/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives. Given a new input, the model produces a self-edit-a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates. Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation. To train the model to produce effective self-edits, we use a reinforcement learning loop with the downstream performance of the updated model as the reward signal. Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model's own generation to control its adaptation process. Experiments on knowledge incorporation and few-shot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation. Our website and code is available at https://jyopari.github.io/posts/seal.