# LLM-Guided Runtime Parameter Optimization for Energy-Efficient Model Inference

arXiv: https://arxiv.org/abs/2604.27032 · HTML: https://mlsanity.com/paper/2604.27032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 5 of 5)

- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference](https://mlsanity.com/paper/2505.09598/) (2025)
- [Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference](https://mlsanity.com/paper/2403.20306/) (2024)
- [Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference](https://mlsanity.com/paper/2303.04673/) (2023)
- [ThinkPatterns-21k: A Systematic Study on the Impact of Thinking Patterns in LLMs](https://mlsanity.com/paper/2503.12918/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have become an integral part of many real-world workflows. However, LLMs consume a lot of energy, which becomes a large concern in the scale of the demand for these tools. As LLMs become integrated into different workflows, different applications have arisen to deal with the challenge of running inference for these tools. This raises another issue of choosing the runtime parameter values for these services in order to minimize the energy consumption. Oftentimes this requires deep knowledge of the application or traditional optimization methods that can take days to find optimal values. In this work, we created a human-in-the-loop flow with LLM-assisted runtime parameter optimization in order to solve this issue. With human-created, specific feedback prompting methods, chat-based LLMs can iteratively find energy-efficient inference parameters faster than traditional search methods. LLMs can also tailor their solutions to different hardware setups and easily take into account other system constraints. The enhanced prompt template was able to converge below the threshold at an average of 3.4 prompts compared to the baseline, which converged in an average of 5.2 prompts, and consistently achieved lower final energy per token. The enhanced prompt template also outperformed Sobol sampling in convergence speed.