mlsanityData to 14 Oct 2026

Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression

arXiv 2505.19433 ↗First posted 26 May 2025cs.LGcited by 16

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Post-training compression reduces the computational and memory costs of large language models (LLMs), enabling resource-efficient deployment. However, existing compression benchmarks only focus on language modeling (e.g., perplexity) and natural language understanding tasks (e.g., GLUE accuracy), ignoring the agentic capabilities - workflow, tool use/function call, long-context understanding and real-world application. We introduce the Agent Compression Benchmark (ACBench), the first comprehensive benchmark for evaluating how compression impacts LLMs' agentic abilities. ACBench spans (1) 12 tasks across 4 capabilities (e.g., WorfBench for workflow generation, Needle-in-Haystack for long-context retrieval), (2) quantization (GPTQ, AWQ) and pruning (Wanda, SparseGPT), and (3) 15 models, including small (Gemma-2B), standard (Qwen2.5 7B-32B), and distilled reasoning LLMs (DeepSeek-R1-Distill). Our experiments reveal compression tradeoffs: 4-bit quantization preserves workflow generation and tool use (1%-3% drop) but degrades real-world application accuracy by 10%-15%. We introduce ERank, Top-k Ranking Correlation and Energy to systematize analysis. ACBench provides actionable insights for optimizing LLM compression in agentic scenarios. The code can be found in https://github.com/pprp/ACBench.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 123 of its 156 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

123 of 156 references.

References · 123

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
38 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  13. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  14. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  15. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  16. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  17. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  18. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  19. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  20. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  21. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  22. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  23. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  24. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
  25. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  26. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  27. 2019Importance Estimation for Neural Network Pruningcited by 541
  28. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  29. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
  30. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  31. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  32. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  33. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
  34. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  35. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  36. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  37. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  38. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveragingcited by 25
  3. 2026VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editingcited by 9
  4. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compressioncited by 8
  5. 2026Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoningcited by 6
  6. 2025FinKario: Event-Enhanced Automated Construction of Financial Knowledge Graphcited by 4
  7. 2025AnTKV: Anchor Token-Aware Sub-Bit Vector Quantization for KV Cache in Large Language Modelscited by 3
  8. 2026Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interactioncited by 2
  9. 2026Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agentscited by 2
  10. 2026Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graphcited by 1
  11. 2026QuantClaw: Precision Where It Matters for OpenClawcited by 1
  12. 2026Evaluating Memory Condensation Strategies for Coding Agents in Data-Driven Scientific Discoverycited by 1
4 more
  1. 2026Which Decisions Low-Bit Quantization Breaks, and How to Predict Themcited by 1
  2. 2026UniComp: A Unified Evaluation of Large Language Model Compression via Pruning, Quantization, and Distillationcited by 0
  3. 2026AGENTQ: Quantization-Conditioned Backdoor Attacks on LLM Agentscited by 0
  4. 2026Quantization Effects on Tool-Failure Recovery Vary Across Prompts and Evaluation Designscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agents
  2. 2023The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models
  3. 2025CompactPrompt: A Unified Pipeline for Prompt Data Compression in LLM Workflows
  4. 2023A Survey on Model Compression for Large Language Models
  5. 2025Semantic Retention and Extreme Compression in LLMs: Can We Have Both?
  6. 2026Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
  7. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple
  8. 2026AgentCompress: Task-Aware Compression for Affordable Large Language Model Agents