# Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv: https://arxiv.org/abs/2607.27275 · HTML: https://mlsanity.com/paper/2607.27275/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 11 of 11)

- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents](https://mlsanity.com/paper/2512.07850/) (2025)
- [Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression](https://mlsanity.com/paper/2505.19433/) (2025)
- [Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels](https://mlsanity.com/paper/2605.15208/) (2026)
- [Secure or Suspect? Investigating Package Hallucinations of Shell Command in Original and Quantized LLMs](https://mlsanity.com/paper/2512.08213/) (2025)

## Built on by (most cited first, 2 of 2)

- [Which Decisions Low-Bit Quantization Breaks, and How to Predict Them](https://mlsanity.com/paper/2608.06564/) (2026)
- [Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designs](https://mlsanity.com/paper/2610.07781/) (2026)

## Abstract (arXiv)

Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most. On τ2-bench, across two open-weight model families in dense and MoE variants and two domains (eight cells, 456 episodes each, at 16-, 8-, and 4-bit weights), quantization indeed looks free on the standard metric. No cell shows a score change that survives multiple-comparison correction, and in the cell that carries the largest process damage, equivalence testing bounds the change within ±7.5 points. The process tells a different story. Quantization amplifies the failure the model already exhibits at full precision (tool-name hallucination in telecom, with the same directional trend in retail entity errors) by up to 2.5× in volume (+17.6 points per task), while creating essentially no new failures. The failure set is the same at every precision (rank correlation ≥ 0.94, 0.18% novel events). The score stays flat because the benchmark's ten-error budget absorbs the extra failures. Shrinking the budget to two errors re-exposes a score gap of 17 points, and it does so only in the one cell where quantization added error volume, exactly as the masking account predicts. A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives. Both diagnostics, the per-channel error rate and success under a shrinking budget, come from logs benchmarks already collect; we suggest reporting them alongside task reward.