mlsanityData to 14 Oct 2026

Prompting for Performance: Exploring LLMs for Configuring Software

arXiv 2507.09790 ↗First posted 13 Jul 2025cs.SE cs.AI cs.PFcited by 1

Not on the map. Why?

Abstract (arXiv)

Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging and requires domain expertise in options and their combinations. On the other hand, machine learning techniques can search vast configuration spaces, but with a high computational cost, since concrete executions of numerous configurations are required. In this exploratory study, we investigate whether large language models (LLMs) can assist in performance-oriented software configuration through prompts. We evaluate several LLMs on tasks including identifying relevant options, ranking configurations, and recommending performant configurations across various configurable systems, such as compilers, video encoders, and SAT solvers. Our preliminary results reveal both positive abilities and notable limitations: depending on the task and systems, LLMs can well align with expert knowledge, whereas hallucinations or superficial reasoning can emerge in other cases. These findings represent a first step toward systematic evaluations and the design of LLM-based solutions to assist with software configuration.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 13 Jul 2025.

Builds on

22 of 44 references.

References · 22

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  5. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  6. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  7. 2019YouTube UGC Dataset for Video Compression Researchcited by 124
  8. 2024Efficient multi-prompt evaluation of LLMscited by 73
  9. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 47
  10. 2018Finding Faster Configurations using FLASHcited by 32
  11. 2024Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4cited by 28
  12. 2017Transfer Learning for Performance Modeling of Configurable Systems: An Exploratory Analysiscited by 20
10 more
  1. 2023KernelGPT: Enhanced Kernel Fuzzing via Large Language Modelscited by 17
  2. 2019Learning Software Configuration Spaces: A Systematic Literature Reviewcited by 14
  3. 2024Configurable Safety Tuning of Language Models with Synthetic Preference Datacited by 14
  4. 2017Transfer Learning for Improving Model Predictions in Highly Configurable Softwarecited by 12
  5. 2023How Effective are Large Language Models in Generating Software Specifications?cited by 12
  6. 2024Large Language Models for Zero Touch Network Configuration Managementcited by 10
  7. 2017Using Bad Learners to find Good Configurationscited by 8
  8. 2021The Interaction between Inputs and Configurations fed to Software Systems: an Empirical Studycited by 6
  9. 2023Configuration Validation with Large Language Modelscited by 4
  10. 2024Code-Survey: An LLM-Driven Methodology for Analyzing Large-Scale Codebasescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Bounded PCTL Model Checking of Large Language Model Outputscited by 1