mlsanityData to 14 Oct 2026

ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function Calling

arXiv 2510.14703 ↗First posted 16 Oct 2025cs.AIcited by 4

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Large language models (LLMs) excel at function calling, but inference scaling has been explored mainly for unstructured generation. We propose an inference-scaling framework for structured outputs that combines fine-grained beam search with ToolPRM, a process reward model scoring each intra-call decision (function name and argument filling). We build the first fine-grained intra-call supervision dataset via function masking, rollout collection, and step-level annotation. ToolPRM outperforms outcome and coarse-grained reward models in predictive accuracy and yields consistent test-time gains on multiple function-calling benchmarks. We further show that structured generation follows ``explore more but retain less'', since early JSON errors are unrecoverable.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 41 of its 60 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

41 of 60 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario17 Jan
  2. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls4 Sep
  3. 2024ToolACE: Winning the Points of LLM Function Calling2 Sep
  4. 2024TinyAgent: Function Calling at the Edge1 Sep
  5. 2024Octopus: On-device language model for function calling of software APIs2 Apr
  6. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  7. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  8. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  11. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  12. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
21 more
  1. 2023Large Language Modelscited by 780
  2. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  3. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  4. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  5. 2023Planning with Large Language Models for Code Generationcited by 182
  6. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  7. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 102
  8. 2025A Survey of AI Agent Protocolscited by 81
  9. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  10. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  11. 2023Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decodingcited by 60
  12. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 45
  13. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Aheadcited by 44
  14. 2025Flexible and Efficient Grammar-Constrained Decodingcited by 43
  15. 2025R-PRM: Reasoning-Driven Process Reward Modelingcited by 38
  16. 2025StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Modelscited by 26
  17. 2025Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoningcited by 23
  18. 2024MemoCRS: Memory-enhanced Sequential Conversational Recommender Systems with Large Language Modelscited by 17
  19. 2025ToolComp: A Multi-Tool Reasoning & Process Supervision Benchmarkcited by 9
  20. 2024FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Rerankingcited by 5
  21. 2025Lightweight Latent Verifiers for Efficient Meta-Generation Strategiescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysiscited by 5
  2. 2026TopoCurate:Modeling Interaction Topology for Tool-Use Agent Trainingcited by 4
  3. 2026Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Argumentscited by 0
  4. 2026Divergent Paths: Learning an Outcome-Predictive Representation of LLM Agent Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Reliable Benchmarking: A Contamination Free, Controllable Evaluation Framework for Multi-step LLM Function Calling
  2. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Models
  3. 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling
  4. 2026Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs
  5. 2026FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation
  6. 2025FunReason: Enhancing Large Language Models' Function Calling via Self-Refinement Multiscale Loss and Automated Data Refinement
  7. 2026Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
  8. 2025A*-Decoding: Token-Efficient Inference Scaling