# Prompting for Performance: Exploring LLMs for Configuring Software

arXiv: https://arxiv.org/abs/2507.09790 · HTML: https://mlsanity.com/paper/2507.09790/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 13 Jul 2025.

## Builds on (most cited first, 22 of 22)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [Efficient multi-prompt evaluation of LLMs](https://mlsanity.com/paper/2405.17202/) (2024)
- [A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks](https://mlsanity.com/paper/2407.12994/) (2024)
- [Finding Faster Configurations using FLASH](https://mlsanity.com/paper/1801.02175/) (2018)
- [Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4](https://mlsanity.com/paper/2401.13810/) (2024)
- [Transfer Learning for Performance Modeling of Configurable Systems: An Exploratory Analysis](https://mlsanity.com/paper/1709.02280/) (2017)
- [KernelGPT: Enhanced Kernel Fuzzing via Large Language Models](https://mlsanity.com/paper/2401.00563/) (2023)
- [Learning Software Configuration Spaces: A Systematic Literature Review](https://mlsanity.com/paper/1906.03018/) (2019)
- [Configurable Safety Tuning of Language Models with Synthetic Preference Data](https://mlsanity.com/paper/2404.00495/) (2024)
- [Transfer Learning for Improving Model Predictions in Highly Configurable Software](https://mlsanity.com/paper/1704.00234/) (2017)
- [How Effective are Large Language Models in Generating Software Specifications?](https://mlsanity.com/paper/2306.03324/) (2023)
- [Large Language Models for Zero Touch Network Configuration Management](https://mlsanity.com/paper/2408.13298/) (2024)
- [Using Bad Learners to find Good Configurations](https://mlsanity.com/paper/1702.05701/) (2017)
- [The Interaction between Inputs and Configurations fed to Software Systems: an Empirical Study](https://mlsanity.com/paper/2112.07279/) (2021)
- [Configuration Validation with Large Language Models](https://mlsanity.com/paper/2310.09690/) (2023)
- [Code-Survey: An LLM-Driven Methodology for Analyzing Large-Scale Codebases](https://mlsanity.com/paper/2410.01837/) (2024)

## Built on by (most cited first, 1 of 1)

- [Bounded PCTL Model Checking of Large Language Model Outputs](https://mlsanity.com/paper/2509.18836/) (2025)

## Abstract (arXiv)

Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging and requires domain expertise in options and their combinations. On the other hand, machine learning techniques can search vast configuration spaces, but with a high computational cost, since concrete executions of numerous configurations are required. In this exploratory study, we investigate whether large language models (LLMs) can assist in performance-oriented software configuration through prompts. We evaluate several LLMs on tasks including identifying relevant options, ranking configurations, and recommending performant configurations across various configurable systems, such as compilers, video encoders, and SAT solvers. Our preliminary results reveal both positive abilities and notable limitations: depending on the task and systems, LLMs can well align with expert knowledge, whereas hallucinations or superficial reasoning can emerge in other cases. These findings represent a first step toward systematic evaluations and the design of LLM-based solutions to assist with software configuration.