mlsanityData to 14 Oct 2026

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv 2607.27275 ↗First posted 29 Jul 2026cs.LG cs.AIcited by 2

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most. On τ2-bench, across two open-weight model families in dense and MoE variants and two domains (eight cells, 456 episodes each, at 16-, 8-, and 4-bit weights), quantization indeed looks free on the standard metric. No cell shows a score change that survives multiple-comparison correction, and in the cell that carries the largest process damage, equivalence testing bounds the change within ±7.5 points. The process tells a different story. Quantization amplifies the failure the model already exhibits at full precision (tool-name hallucination in telecom, with the same directional trend in retail entity errors) by up to 2.5× in volume (+17.6 points per task), while creating essentially no new failures. The failure set is the same at every precision (rank correlation ≥ 0.94, 0.18% novel events). The score stays flat because the benchmark's ten-error budget absorbs the extra failures. Shrinking the budget to two errors re-exposes a score gap of 17 points, and it does so only in the one cell where quantization added error volume, exactly as the masking account predicts. A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives. Both diagnostics, the per-channel error rate and success under a shrinking budget, come from logs benchmarks already collect; we suggest reporting them alongside task reward.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 11 of its 16 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

11 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  2. 2023Gorilla: Large Language Model Connected with Massive APIs24 May

Other references · 9

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  5. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  6. 2025SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agentscited by 21
  7. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 16
  8. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levelscited by 6
  9. 2025Secure or Suspect? Investigating Package Hallucinations of Shell Command in Original and Quantized LLMscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
  2. 2026From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
  3. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  4. 2026Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bit
  5. 2025Why Do Some Inputs Break Low-Bit LLM Quantization?
  6. 2026Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression
  7. 2023Intriguing Properties of Quantization at Scale
  8. 2026Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs