mlsanityData to 14 Oct 2026

Compiling Expert Procedures into Verified Programs: When a Small Model Is Enough

arXiv 2610.19148 ↗First posted 13 Oct 2026cs.AI cs.SCcited by 0

Not on the map. Why?

Abstract (arXiv)

In procedural tool-use domains, how much of the gap between base models is procedure rather than capability? We build an explicit program from a teacher's traces and run it in front of unchanged small models, including an 8B model. The construction, PLVR , takes the reasoning traces of a teacher, Kimi-K3, clusters their segments to locate recurring operations, has a frontier model author one typed primitive per operation, each run by a small frozen model and assembles the program by A* search over a mined rule library, under a reference-action objective. At inference the program takes one of two forms: for troubleshooting and tool search it produces the chain of thought the student would otherwise generate for itself, and the student acts on it; for multi-turn tool matching the student proposes each action and the program checks, repairs, or retries it before it goes out. On the official tau2-Bench Retail and Telecom test sets and on LiveMCPBench, with three Nemotron base models and the same program for each, the answer depends on the domain. On Telecom, Granite-4.2-8B as the student goes from 43.0% to 91.2% under the program, and every Nemotron student reaches 93.3-94.4%, against at most 58.2% for fine-tuning on the same traces, prompt optimisation of the student, or two token-matched baselines. With Granite-4.2-8B as the student, the program is within 2 to 3 points of the Nemotron students on Telecom and within 3 on Retail, and 3 to 13 points below them on LiveMCPBench: where the domain is procedural an 8B student is enough, and on tool search the student still matters. On Retail and LiveMCPBench the program is competitive with the best baseline. The spread between base models shrinks from about 7 points to about 2, averaged over the suites, with no weights changed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 35 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

10 of 35 references.

References · 10

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  5. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  6. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 77
  7. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  8. 2024Proof of Thought : Neurosymbolic Program Synthesis allows Robust and Interpretable Reasoningcited by 19
  9. 2025Efficient PRM Training Data Synthesis via Formal Verificationcited by 8
  10. 2026SEVerA: Verified Synthesis of Self-Evolving Agentscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents
  2. 2026Procedural Knowledge at Scale Improves Reasoning
  3. 2025Generating Verifiable Chain of Thoughts from Exection-Traces
  4. 2025L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution
  5. 2026Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime for Procedural LLM Agents
  6. 2025Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation
  7. 2026Methods for Formal Verification of Agent Skills: Three Layers Toward a Mechanically Checkable Capability-Containment Proof
  8. 2026Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments