mlsanityData to 7 Oct 2026

Balancing Sustainability And Performance: The Role Of Small-Scale LLMs In Agentic Artificial Intelligence Systems

arXiv 2601.19311 ↗First posted 27 Jan 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As large language models become integral to agentic artificial intelligence systems, their energy demands during inference may pose significant sustainability challenges. This study investigates whether deploying smaller-scale language models can reduce energy consumption without compromising responsiveness and output quality in a multi-agent, real-world environments. We conduct a comparative analysis across language models of varying scales to quantify trade-offs between efficiency and performance. Results show that smaller open-weights models can lower energy usage while preserving task quality. Building on these findings, we propose practical guidelines for sustainable artificial intelligence design, including optimal batch size configuration and computation resource allocation. These insights offer actionable strategies for developing scalable, environmentally responsible artificial intelligence systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 23 of its 30 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

23 of 30 references.

References · 23

  1. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  2. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  3. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  4. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  5. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  6. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  7. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
  8. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 119
  9. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 73
  10. 2024"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantizationcited by 34
  11. 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 31
  12. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 31
11 more
  1. 2025Measuring the environmental impact of delivering AI at Google Scalecited by 31
  2. 2025Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Servingcited by 21
  3. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scalingcited by 19
  4. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Computecited by 14
  5. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giantcited by 12
  6. 2025TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inferencecited by 11
  7. 2025On Evaluating Performance of LLM Inference Serving Systemscited by 9
  8. 2025Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspectivecited by 8
  9. 2025Latency and Token-Aware Test-Time Computecited by 8
  10. 2024An exploration of the effect of quantisation on energy consumption and inference time of StarCoder2cited by 4
  11. 2025Exploring the sustainable scaling of AI dilemma: A projective study of corporations' AI environmental impactscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Addressing the sustainable AI trilemma: a case study on LLM agents and RAG
  2. 2026Environmental Impact of Generative and Agentic AI: An in-Depth Analysis and Green Solutions
  3. 2024Practical Considerations for Agentic LLM Systems
  4. 2024Leveraging Hybrid Intelligence Towards Sustainable and Energy-Efficient Machine Learning
  5. 2026Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey
  6. 2024Coalitions of Large Language Models Increase the Robustness of AI Agents
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  8. 2026EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents