mlsanityData to 7 Oct 2026

Addressing the sustainable AI trilemma: a case study on LLM agents and RAG

arXiv 2501.08262 ↗First posted 14 Jan 2025cs.CYcited by 1

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Large language models (LLMs) have demonstrated significant capabilities, but their widespread deployment and more advanced applications raise critical sustainability challenges, particularly in inference energy consumption. We propose the concept of the Sustainable AI Trilemma, highlighting the tensions between AI capability, digital equity, and environmental sustainability. Through a systematic case study of LLM agents and retrieval-augmented generation (RAG), we analyze the energy costs embedded in memory module designs and introduce novel metrics to quantify the trade-offs between energy consumption and system performance. Our experimental results reveal significant energy inefficiencies in current memory-augmented frameworks and demonstrate that resource-constrained environments face disproportionate efficiency penalties. Our findings challenge the prevailing LLM-centric paradigm in agent design and provide practical insights for developing more sustainable AI systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 73 references that are in the map. First posted on arXiv 14 Jan 2025.

Builds on

37 of 73 references.

References · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  7. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  9. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  10. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  11. 2024Understanding the planning of LLM agents: A surveycited by 414
  12. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
25 more
  1. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  2. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  3. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  4. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 157
  5. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  6. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  7. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  8. 2022Holistic Evaluation of Language Modelscited by 115
  9. 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
  10. 2023MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversationcited by 94
  11. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  12. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 71
  13. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  14. 2023RET-LLM: Towards a General Read-Write Memory for Large Language Modelscited by 64
  15. 2023Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memorycited by 64
  16. 2024LLM With Tools: A Surveycited by 60
  17. 2023Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Rankingcited by 44
  18. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  19. 2023PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adaptercited by 38
  20. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMscited by 27
  21. 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 27
  22. 2024Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generationcited by 7
  23. 2024ARAGOG: Advanced RAG Output Gradingcited by 7
  24. 2024Whose ChatGPT? Unveiling Real-World Educational Inequalities Introduced by Large Language Modelscited by 5
  25. 2024Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloudcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026On the Effectiveness of Proposed Techniques to Reduce Energy Consumption in RAG Systems: A Controlled Experimentcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Balancing Sustainability And Performance: The Role Of Small-Scale LLMs In Agentic Artificial Intelligence Systems
  2. 2026Environmental Impact of Generative and Agentic AI: An in-Depth Analysis and Green Solutions
  3. 2026EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents
  4. 2026The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products
  5. 2026Sustainable LLM Inference using Context-Aware Model Switching
  6. 2024Practical Considerations for Agentic LLM Systems
  7. 2026From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling
  8. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems