# Energy-Aware Code Generation with LLMs: Benchmarking Small vs. Large Language Models for Sustainable AI Programming

arXiv: https://arxiv.org/abs/2508.08332 · HTML: https://mlsanity.com/paper/2508.08332/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 10 Aug 2025.

## Builds on (most cited first, 20 of 20)

- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [ReCode: Robustness Evaluation of Code Generation Models](https://mlsanity.com/paper/2212.10264/) (2022)
- [EffiBench: Benchmarking the Efficiency of Automatically Generated Code](https://mlsanity.com/paper/2402.02037/) (2024)
- [Compute and Energy Consumption Trends in Deep Learning Inference](https://mlsanity.com/paper/2109.05472/) (2021)
- [How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark](https://mlsanity.com/paper/2406.06647/) (2024)
- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [Mercury: A Code Efficiency Benchmark for Code Large Language Models](https://mlsanity.com/paper/2402.07844/) (2024)
- [Holistically Evaluating the Environmental Impact of Creating Language Models](https://mlsanity.com/paper/2503.05804/) (2025)
- [On Evaluating the Efficiency of Source Code Generated by LLMs](https://mlsanity.com/paper/2404.06041/) (2024)
- [A Controlled Experiment on the Energy Efficiency of the Source Code Generated by Code Llama](https://mlsanity.com/paper/2405.03616/) (2024)
- [AI-Powered, But Power-Hungry? Energy Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2502.02412/) (2025)
- [Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation](https://mlsanity.com/paper/2403.03344/) (2024)
- [Green My LLM: Studying the key factors affecting the energy consumption of code assistants](https://mlsanity.com/paper/2411.11892/) (2024)
- [Generating Energy-efficient code with LLMs](https://mlsanity.com/paper/2411.10599/) (2024)
- [Carbon Footprint Evaluation of Code Generation through LLM as a Service](https://mlsanity.com/paper/2504.01036/) (2025)
- [Comparative Analysis of Carbon Footprint in Manual vs. LLM-Assisted Code Development](https://mlsanity.com/paper/2505.04521/) (2025)
- [Evaluating the Energy-Efficiency of the Code Generated by LLMs](https://mlsanity.com/paper/2505.20324/) (2025)
- [Sustainability via LLM Right-sizing](https://mlsanity.com/paper/2504.13217/) (2025)
- [Large Language Models for Energy-Efficient Code: Emerging Results and Future Directions](https://mlsanity.com/paper/2410.09241/) (2024)
- [Can We Make Code Green? Understanding Trade-Offs in LLMs vs. Human Code Optimizations](https://mlsanity.com/paper/2503.20126/) (2025)

## Built on by (most cited first, 2 of 2)

- [Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learning](https://mlsanity.com/paper/2607.04577/) (2026)
- [Improving the Energy-Efficiency of the Code Generated by LLMs through Effective Prompting](https://mlsanity.com/paper/2610.02571/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are widely used for code generation. However, commercial models like ChatGPT require significant computing power, which leads to high energy use and carbon emissions. This has raised concerns about their environmental impact. In this study, we evaluate open-source Small Language Models (SLMs) trained explicitly for code generation and compare their performance and energy efficiency against large LLMs and efficient human-written Python code. The goal is to investigate whether SLMs can match the performance of LLMs on certain types of programming problems while producing more energy-efficient code. We evaluate 150 coding problems from LeetCode, evenly distributed across three difficulty levels: easy, medium, and hard. Our comparison includes three small open-source models, StableCode-3B, StarCoderBase-3B, and Qwen2.5-Coder-3B-Instruct, and two large commercial models, GPT-4.0 and DeepSeek-Reasoner. The generated code is evaluated using four key metrics: run-time, memory usage, energy consumption, and correctness. We use human-written solutions as a baseline to assess the quality and efficiency of the model-generated code. Results indicate that LLMs achieve the highest correctness across all difficulty levels, but SLMs are often more energy-efficient when their outputs are correct. In over 52% of the evaluated problems, SLMs consumed the same or less energy than LLMs.