# The Price of Prompting: Profiling Energy Use in Large Language Models Inference

arXiv: https://arxiv.org/abs/2407.16893 · HTML: https://mlsanity.com/paper/2407.16893/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 69 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model](https://mlsanity.com/paper/2211.02001/) (2022)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement](https://mlsanity.com/paper/2402.14658/) (2024)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference](https://mlsanity.com/paper/2310.03003/) (2023)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT](https://mlsanity.com/paper/2302.10198/) (2023)
- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [MarioGPT: Open-Ended Text2Level Generation through Large Language Models](https://mlsanity.com/paper/2302.05981/) (2023)
- [Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference](https://mlsanity.com/paper/2403.20306/) (2024)
- [Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions](https://mlsanity.com/paper/2305.15083/) (2023)
- [Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads](https://mlsanity.com/paper/2407.00010/) (2024)
- [Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUs](https://mlsanity.com/paper/2002.07795/) (2020)
- [GPT4All: An Ecosystem of Open Source Compressed Language Models](https://mlsanity.com/paper/2311.04931/) (2023)
- [The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models](https://mlsanity.com/paper/2406.19358/) (2024)

## Built on by (most cited first, 23 of 23)

- [How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference](https://mlsanity.com/paper/2505.09598/) (2025)
- [A Survey of Research in Large Language Models for Electronic Design Automation](https://mlsanity.com/paper/2501.09655/) (2025)
- [Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency](https://mlsanity.com/paper/2504.03360/) (2025)
- [A Survey of Small Language Models](https://mlsanity.com/paper/2410.20011/) (2024)
- [Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k](https://mlsanity.com/paper/2506.08479/) (2025)
- [The Rise of Small Language Models in Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2504.17119/) (2025)
- [Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures](https://mlsanity.com/paper/2604.09048/) (2026)
- [Green MLOps to Green GenOps: An Empirical Study of Energy Consumption in Discriminative and Generative AI Operations](https://mlsanity.com/paper/2503.23934/) (2025)
- [The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products](https://mlsanity.com/paper/2604.00053/) (2026)
- [MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications](https://mlsanity.com/paper/2411.18915/) (2024)
- [Performance Trade-offs of Optimizing Small Language Models for E-Commerce](https://mlsanity.com/paper/2510.21970/) (2025)
- [Characterizing and Understanding Energy Footprint and Efficiency of Small Language Model on Edges](https://mlsanity.com/paper/2511.11624/) (2025)
- [KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving](https://mlsanity.com/paper/2604.16682/) (2026)
- [Small Language Models: Architectures, Techniques, Evaluation, Problems and Future Adaptation](https://mlsanity.com/paper/2505.19529/) (2025)
- [Energy-Aware Routing to Large Reasoning Models](https://mlsanity.com/paper/2601.00823/) (2025)
- [Emissions and Performance Trade-off Between Small and Large Language Models](https://mlsanity.com/paper/2601.08844/) (2025)
- [Optimising for Energy Efficiency and Performance in Machine Learning](https://mlsanity.com/paper/2601.08991/) (2026)
- [SEALing the Gap: A Reference Framework for LLM Inference Carbon Estimation via Multi-Benchmark Driven Embodiment](https://mlsanity.com/paper/2603.02949/) (2026)
- [Improving Multi-turn Dialogue Consistency with Self-Recall Thinking](https://mlsanity.com/paper/2605.15102/) (2026)
- [From Caveman to Expert Analyst: Energy Consumption of Variable LLM Tasks](https://mlsanity.com/paper/2608.12350/) (2026)
- [Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures](https://mlsanity.com/paper/2608.25096/) (2026)
- [How Do Prompt Variations Affect Energy Consumption in On-Device LLMs?](https://mlsanity.com/paper/2609.01798/) (2026)
- [Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms](https://mlsanity.com/paper/2610.20136/) (2026)

## Abstract (arXiv)

In the rapidly evolving realm of artificial intelligence, deploying large language models (LLMs) poses increasingly pressing computational and environmental challenges. This paper introduces MELODI - Monitoring Energy Levels and Optimization for Data-driven Inference - a multifaceted framework crafted to monitor and analyze the energy consumed during LLM inference processes. MELODI enables detailed observations of power consumption dynamics and facilitates the creation of a comprehensive dataset reflective of energy efficiency across varied deployment scenarios. The dataset, generated using MELODI, encompasses a broad spectrum of LLM deployment frameworks, multiple language models, and extensive prompt datasets, enabling a comparative analysis of energy use. Using the dataset, we investigate how prompt attributes, including length and complexity, correlate with energy expenditure. Our findings indicate substantial disparities in energy efficiency, suggesting ample scope for optimization and adoption of sustainable measures in LLM deployment. Our contribution lies not only in the MELODI framework but also in the novel dataset, a resource that can be expanded by other researchers. Thus, MELODI is a foundational tool and dataset for advancing research into energy-conscious LLM deployment, steering the field toward a more sustainable future.