Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designs
On the map: LLM agents and tool use → Tool learning and function calling
Abstract (arXiv)
Post-training quantization reduces the cost of deploying language-model agents, but its effect on recovery from temporary tool failures can depend on how recovery is evaluated. We compare 8-bit and 4-bit variants of Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct on twenty deterministic tool-use tasks and five prompts. The 8-bit-4-bit recovery comparison changes direction across prompts and evaluation targets. On tasks that both variants complete without faults under the same prompt, the difference ranges from 0 to +20.2 percentage points for Llama and from -50.0 to +35.0 points for Qwen. Full-pipeline point estimates favor 8-bit Llama under all five prompts, whereas the Qwen comparison changes direction across prompts. The evaluation target can also reverse the result. For Llama under one prompt, scoring each variant only on its own clean-passing tasks favors 4-bit by 17.5 points; scoring the same tasks for both variants gives no difference, while scoring the full pipeline favors 8-bit by 28.3 points. Executor leniency is a third such choice. Rescoring the same logs with strict output parsing, which 8-bit Llama violates far more often than 4-bit Llama under that prompt, turns that +28.3 into -15.0 while leaving Qwen essentially unchanged. These findings show that one prompt, one screened task set, and one scoring policy do not establish a stable conclusion about quantized-agent robustness. Evaluations should compare variants on matched tasks, report full-pipeline success for deployment decisions, state the scoring policy, and quantify uncertainty across tasks rather than injected fault sites.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. First posted on arXiv 6 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Quantization Thresholds Replicate, Failure Modes Do Not: A Three-Model Study of Agentic Tool Use in Polish from 8-bit to 2-bit
- 2026Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
- 2026Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents
- 2026Loud Failures, Quiet Failures: Fault Detection and Recovery in Tool-Using Language Model Agents
- 2026Joint Optimization of Tool Creation and Use for Large Language Model Agents
- 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Cases
- 2026Choosing Before Acting: Comparative Value Estimation for Long-Horizon Tool-Use Agents
- 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts