mlsanityData to 14 Oct 2026

Scaffold-Mediated Post-Training: Co-Evolving Model Parameters and Procedural Scaffold Graphs

arXiv 2608.05156 ↗First posted 22 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training of large language models optimizes only parameters, while inference-time procedural scaffolds are typically designed independently of parameter training. This disconnect makes it difficult to automatically acquire and internalize complex strategies. We propose scaffold-mediated post-training: procedural scaffolds are organized into an evolvable graph structure that co-evolves with model parameters through discovery, distillation, and dynamic recompilation. We instantiate this paradigm as Skill Training. On FeatureBench, automatically discovered skills improve the passed rate by 8.1pp, and after progressive distillation the model still achieves a 27.7% passed rate without any external scaffold (distillation retention rate 85.2%, defined as post-distillation / with-skill passed rate), significantly outperforming standard SFT on the same data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

18 of 21 references.

References · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  6. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  7. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  9. 2023Large Language Models as Optimizerscited by 864
  10. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  11. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
6 more
  1. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  2. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
  3. 2022Learning by Distilling Contextcited by 138
  4. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 121
  5. 2025Reinforcement Learning for Self-Improving Agent with Skill Librarycited by 100
  6. 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Developmentcited by 34

Built on by

Not cited yet.