mlsanityData to 14 Oct 2026

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity

arXiv 2609.17515 ↗First posted 15 Sep 2026cs.CLcited by 0

On the map: LLM agents and tool use → Tool learning and function calling

also in 1 more
Abstract (arXiv)

Pruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts (MoE) architectures, together with depth, width, hybrid, and expert pruning methods. After post-pruning supervised fine-tuning (SFT), we evaluate more than 19,500 instances from three smart-home datasets. Beyond aggregate task accuracy, we characterize degradation along two dimensions: action components (i.e., operation, device, argument, and value) and task complexity. Our results show that dense models have narrow safe pruning regions followed by sharp degradation, while MoE models tolerate substantially more pruning. Pruning degrades grounded specificity before schema-level intent, and aggressive dense pruning can induce systematic over-refusal. These findings highlight the importance of evaluating pruning beyond aggregate accuracy when selecting pruned LLMs for reliable tool execution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 10 of its 15 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

10 of 15 references.

References · 10

  1. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  2. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
  3. 2025REAP the Experts: Why Pruning Prevails for One-Shot MoE compressioncited by 44
  4. 2025HomeBench: Evaluating LLMs in Smart Homes with Valid and Invalid Instructions Across Single and Multiple Devicescited by 15
  5. 2025SimuHome: A Temporal- and Environment-Aware Benchmark for Smart Home LLM Agentscited by 15
  6. 2023Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMscited by 13
  7. 2025MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoEcited by 13
  8. 20252SSP: A Two-Stage Framework for Structured Pruning of LLMscited by 12
  9. 2026SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homescited by 4
  10. 2026MIST: Multimodal Interactive Speech-based Tool-calling Conversational Assistants for Smart Homescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Reducing Tool Hallucination via Reliability Alignment
  2. 2026Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AI
  3. 2025Automatic Pruning Discovery for Large Language Models
  4. 2021Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
  5. 2026From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
  6. 2025ToLeaP: Rethinking Development of Tool Learning with Large Language Models
  7. 2025Context Discipline and Performance Correlation: Analyzing LLM Performance and Quality Degradation Under Varying Context Lengths
  8. 2025SMART: Self-Aware Agent for Tool Overuse Mitigation