# Towards Green AI: Decoding the Energy of LLM Inference in Software Development

arXiv: https://arxiv.org/abs/2602.05712 · HTML: https://mlsanity.com/paper/2602.05712/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 19 of 19)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [A Systematic Review of Green AI](https://mlsanity.com/paper/2301.11047/) (2023)
- [Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference](https://mlsanity.com/paper/2403.20306/) (2024)
- [Energy Considerations of Large Language Model Inference and Efficiency Optimizations](https://mlsanity.com/paper/2504.17674/) (2025)
- [An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project](https://mlsanity.com/paper/2401.16186/) (2024)
- [Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling](https://mlsanity.com/paper/2501.08219/) (2025)
- [AI-Powered, But Power-Hungry? Energy Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2502.02412/) (2025)
- [Generating Energy-Efficient Code via Large-Language Models -- Where are we now?](https://mlsanity.com/paper/2509.10099/) (2025)
- [SPAD: Specialized Prefill and Decode Hardware for Disaggregated LLM Inference](https://mlsanity.com/paper/2510.08544/) (2025)
- [AI in Software Engineering: Perceived Roles and Their Impact on Adoption](https://mlsanity.com/paper/2504.20329/) (2025)
- [SENAI: Towards Software Engineering Native Generative Artificial Intelligence](https://mlsanity.com/paper/2503.15282/) (2025)
- [Tu(r)ning AI Green: Exploring Energy Efficiency Cascading with Orthogonal Optimizations](https://mlsanity.com/paper/2506.18289/) (2025)
- [Smart but Costly? Benchmarking LLMs on Functional Accuracy and Energy Efficiency](https://mlsanity.com/paper/2511.07698/) (2025)

## Built on by (most cited first, 2 of 2)

- [Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations](https://mlsanity.com/paper/2607.09172/) (2026)
- [GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon](https://mlsanity.com/paper/2608.28667/) (2026)

## Abstract (arXiv)

Context: AI-assisted tools are increasingly integrated into software development workflows, but their reliance on large language models (LLMs) introduces substantial computational and energy costs. Understanding and reducing the energy footprint of LLM inference is therefore essential for sustainable software development. Objective: In this study, we conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state. Method: We investigate six 6B-7B and four 3B-4B transformer-based models, evaluating them on code-centric benchmarks HumanEval for code generation and LongBench for code understanding. Results: Our findings show that, within both parameter groups, models exhibit distinct energy patterns across phases. Furthermore, we observed that increases in prefill cost amplify the energy cost per token during decoding, with amplifications ranging from 1.3% to 51.8% depending on the model. Lastly, three out of ten models demonstrate babbling behavior, adding excessive content to the output that unnecessarily inflates energy consumption. We implemented babbling suppression for code generation, achieving energy savings ranging from 44% to 89% without affecting generation accuracy. Conclusion: These findings show that prefill costs influence decoding, which dominates energy consumption, and that babbling suppression can yield up to 89% energy savings. Reducing inference energy therefore requires both mitigating babbling behavior and limiting impact of prefill on decoding.