mlsanityData to 7 Oct 2026

From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis

arXiv 2607.24459 ↗First posted 27 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models increasingly solve scientific-computing tasks, but executable feedback from one problem rarely becomes durable capability on subsequent problems. We study scientific-computing experience consolidation: converting verified runtime experience into transferable procedural knowledge and persistent model improvement. This setting presents two challenges: trajectory-derived artifacts may encode source-specific repairs rather than cross-task computational mechanisms; and a weaker target model may be unable to operationalize an otherwise valid abstract procedure - an abstraction-execution gap. We introduce SciConsolidate, which contrasts verified successes and failures to induce cross-task procedures, selects them through a development-validation gate, and uses failure-informed, answer-free query synthesis to expand the consolidation data without requiring pre-existing reference answers. Because the target model may not directly execute these abstractions, a stronger model concretizes them into executable code supervision for standard, procedure-free SFT; a matched no-procedure teacher branch isolates the value of procedural guidance. On SciCode, runtime procedure injection improves Qwen3.6-27B by +3.85/+6.26 sub-step/main-problem points, but yields almost no aggregate main-problem gain for Qwen3.5-9B, providing operational evidence of the abstraction-execution gap. After procedure-guided concretization, the 9B student improves under procedure-free deployment by +3.89/+6.25 points over the no-procedure SFT control and by +5.62/+11.25 over the original 9B model. These results establish an experience-to-capability pathway for scientific computing and provide a practical starting point for scaling self-improving scientific assistance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

35 of 50 references.

References · 35

  1. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  2. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  3. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  4. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  5. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 172
  6. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  7. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  8. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  9. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
  10. 2025Inducing Programmatic Skills for Agentic Taskscited by 103
  11. 2025Memp: Exploring Agent Procedural Memorycited by 88
  12. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
23 more
  1. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  2. 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scalecited by 65
  3. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  4. 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Taskscited by 45
  5. 2025ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Codecited by 34
  6. 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learningcited by 28
  7. 2026SkillGrad: Optimizing Agent Skills Like Gradient Descentcited by 23
  8. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
  9. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 18
  10. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 14
  11. 2026SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineeringcited by 14
  12. 2026SkillEvolver: Skill Learning as a Meta-Skillcited by 12
  13. 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skillscited by 9
  14. 2026Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agentscited by 8
  15. 2025FEM-Bench: A Structured Scientific Reasoning Benchmark for Evaluating Code-Generating LLMscited by 7
  16. 2026Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agentscited by 7
  17. 2026SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Usecited by 7
  18. 2026Recursive Self-Evolving Agents via Held-Out Selectioncited by 6
  19. 2025CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamicscited by 5
  20. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learningcited by 5
  21. 2026Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chainscited by 3
  22. 2026SkillWiki: A Living Knowledge Infrastructure for Agent Skillscited by 2
  23. 2026OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillationcited by 1

Built on by

Not cited yet.