# On the Effectiveness of Proposed Techniques to Reduce Energy Consumption in RAG Systems: A Controlled Experiment

arXiv: https://arxiv.org/abs/2601.02522 · HTML: https://mlsanity.com/paper/2601.02522/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 22 of 22)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://mlsanity.com/paper/2310.07240/) (2023)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [BM25S: Orders of magnitude faster lexical search via eager sparse scoring](https://mlsanity.com/paper/2407.03618/) (2024)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text](https://mlsanity.com/paper/2410.07590/) (2024)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Principled approach to the selection of the embedding dimension of networks](https://mlsanity.com/paper/2004.09928/) (2020)
- [A Synthesis of Green Architectural Tactics for ML-Enabled Systems](https://mlsanity.com/paper/2312.09610/) (2023)
- [RankLLM: A Python Package for Reranking with LLMs](https://mlsanity.com/paper/2505.19284/) (2025)
- [iRAG: Advancing RAG for Videos with an Incremental Approach](https://mlsanity.com/paper/2404.12309/) (2024)
- [Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control](https://mlsanity.com/paper/2502.12145/) (2025)
- [Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking](https://mlsanity.com/paper/2505.19310/) (2025)
- [Addressing the sustainable AI trilemma: a case study on LLM agents and RAG](https://mlsanity.com/paper/2501.08262/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Global Landscape of Environmental AI Regulation: From the Cost of Reasoning to a Right to Green AI](https://mlsanity.com/paper/2603.00068/) (2026)
- [Where Should Agents Live? Energy-Memory Characterization of Agentic AI for the Edge-Cloud Continuum](https://mlsanity.com/paper/2609.18283/) (2026)

## Abstract (arXiv)

The rising energy demands of machine learning (ML), e.g., implemented in popular variants like retrieval-augmented generation (RAG) systems, have raised significant concerns about their environmental sustainability. While previous research has proposed green tactics for ML-enabled systems, their empirical evaluation within RAG systems remains largely unexplored. This study presents a controlled experiment investigating five practical techniques aimed at reducing energy consumption in RAG systems. Using a production-like RAG system developed at our collaboration partner, the Software Improvement Group, we evaluated the impact of these techniques on energy consumption, latency, and accuracy. Through a total of 9 configurations spanning over 200 hours of trials using the CRAG dataset, we reveal that techniques such as increasing similarity retrieval thresholds, reducing embedding sizes, applying vector indexing, and using a BM25S reranker can significantly reduce energy usage, up to 60% in some cases. However, several techniques also led to unacceptable accuracy decreases, e.g., by up to 30% for the indexing strategies. Notably, finding an optimal retrieval threshold and reducing embedding size substantially reduced energy consumption and latency with no loss in accuracy, making these two techniques truly energy-efficient. We present the first comprehensive, empirical study on energy-efficient design techniques for RAG systems, providing guidance for developers and researchers aiming to build sustainable RAG applications.