mlsanityData to 14 Oct 2026

Repair, Not Improvement: Decomposing Constrained Decoding in Tool-Call Abstention

arXiv 2608.13959 ↗First posted 14 Aug 2026cs.CLcited by 4

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

A tool-calling router has to pick the right tool when one applies and decline when none does. Restricting the decoder to a grammar over the tool names is the standard remedy for the first, and on small models it buys a large accuracy gain. Recent work separates the loss caused by asking for a format from the loss caused by enforcing it at decode time. The second is small, which made enforcement look nearly free. The same work declines to extend that to function calling, where a constraint decides which answers exist rather than how one is written. Declining to call anything is the answer it most easily removes, and the one a router can least afford to lose. A grammar decides which tokens may be emitted and where generation stops, and a two-condition design charges both to the restriction. We therefore run three conditions over one prompt: free generation, generation stopped at the first line, and both applied together. We evaluate open-weight models from 0.6B to 4B on the same items in English and Korean, comparing the languages item by item. The two-condition contrast is negative on abstention accuracy in four of six cells with intervals excluding zero, and positive in none, costing -29.5 points at worst. On the smallest model in Korean the stop costs -20.0 points, the restriction returns +19.5, and together they leave -0.5. What the restriction gives back is readable output, not judgment. Of the 698 abstentions it repairs, 545 had no readable answer at all and 0 were correct decisions the scoring rule rejected. On items that do need a tool the contrast is positive throughout, and abstention is reported first because it is the registered measure. Both preregistered claims about language fail: Korean does not lose more of the abstentions it holds without the constraint, and the removed mass does not explain what does.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 12 of its 20 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

12 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraints24 Jun
  2. 2026Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models8 Jan
  3. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct

Other references · 9

  1. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  2. 2021Preregistering NLP Researchcited by 25
  3. 2026Mitigating Bias in Locally Constrained Decoding via Tractable Proposalscited by 8
  4. 2026The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decodingcited by 7
  5. 2025Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Modelscited by 6
  6. 2026The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Modelscited by 4
  7. 2025(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decodingcited by 1
  8. 2026From Hallucination to Structure Snowballing: The Alignment Tax of Constrained Decoding in LLM Reflectioncited by 1
  9. 2026Think in English, Answer in Korean: Efficient Adaptation of Multilingual Tool-Using Agentscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tool Calling is Linearly Readable and Steerable in Language Models
  2. 2026ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents
  3. 2026Thinking Before Constraining: A Unified Decoding Framework for Large Language Models
  4. 2026When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering
  5. 2026The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs
  6. 2026Closed-World Resolution Against Tool Hallucination in LLM Agents
  7. 2026Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax
  8. 2026Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap