# On Evaluating the Efficiency of Source Code Generated by LLMs

arXiv: https://arxiv.org/abs/2404.06041 · HTML: https://mlsanity.com/paper/2404.06041/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2024.

## Built on by (most cited first, 21 of 21)

- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark](https://mlsanity.com/paper/2406.06647/) (2024)
- [EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization](https://mlsanity.com/paper/2405.15189/) (2024)
- [ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?](https://mlsanity.com/paper/2407.14044/) (2024)
- [EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2505.13004/) (2025)
- [GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents](https://mlsanity.com/paper/2505.23671/) (2025)
- [EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning](https://mlsanity.com/paper/2410.10209/) (2024)
- [Measuring the Influence of Incorrect Code on Test Generation](https://mlsanity.com/paper/2409.09464/) (2024)
- [LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctness](https://mlsanity.com/paper/2502.18489/) (2025)
- [A Problem-Oriented Perspective and Anchor Verification for Code Optimization](https://mlsanity.com/paper/2406.11935/) (2024)
- [Generating Energy-efficient code with LLMs](https://mlsanity.com/paper/2411.10599/) (2024)
- [Evaluating the Energy-Efficiency of the Code Generated by LLMs](https://mlsanity.com/paper/2505.20324/) (2025)
- [PerfCoder: Large Language Models for Interpretable Code Performance Optimization](https://mlsanity.com/paper/2512.14018/) (2025)
- [Energy-Aware Code Generation with LLMs: Benchmarking Small vs. Large Language Models for Sustainable AI Programming](https://mlsanity.com/paper/2508.08332/) (2025)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)
- [Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code](https://mlsanity.com/paper/2605.05267/) (2026)
- [Unlocking LLM Code Correction with Iterative Feedback Loops](https://mlsanity.com/paper/2606.17514/) (2026)
- [Do AI Models Dream of Faster Code? An Empirical Study on LLM-Proposed Performance Improvements in Real-World Software](https://mlsanity.com/paper/2510.15494/) (2025)
- [Rethinking Code Performance Benchmarks for LLMs](https://mlsanity.com/paper/2607.07619/) (2026)
- [Improving the Energy-Efficiency of the Code Generated by LLMs through Effective Prompting](https://mlsanity.com/paper/2610.02571/) (2026)
- [Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolution](https://mlsanity.com/paper/2610.13563/) (2026)

## Abstract (arXiv)

Recent years have seen the remarkable capabilities of large language models (LLMs) for code generation. Different from existing work that evaluate the correctness of the code generated by LLMs, we propose to further evaluate its efficiency. More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming. First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP. Then, we choose a set of programming problems from the online judge platform LeetCode to conduct a more difficult evaluation. Finally, we explore several prompts that would enable LLMs to generate more efficient code.