# PromptResponse: Optimizing Prompts for LLM Coding Tasks

arXiv: https://arxiv.org/abs/2608.21074 · HTML: https://mlsanity.com/paper/2608.21074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 8 of 8)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [Efficacy of Synthetic Data as a Benchmark](https://mlsanity.com/paper/2409.11968/) (2024)
- [PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation](https://mlsanity.com/paper/2406.18528/) (2024)
- [Prompt-Hacking: The New p-Hacking?](https://mlsanity.com/paper/2504.14571/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to input prompt variations. This paper presents unicodex00ABPromptResponseunicodex00BB, a controlled study examining how formatting and LLM-based tuning of coding task prompts affect the resulting code's performance, efficiency, and stability. Using five semantically identical yet syntactically distinct variants of the HumanEval datasetunicodex2014baseline, JSON, Markdown, YAML, and an LLM-tuned versionunicodex2014we had GPT-4o solve its coding problems over 8200unicodex00A0executions. Our results show that consistent formattingunicodex2014especially JSONunicodex2014improves generation efficiency and syntactic stability, with minor gains in task performance. Conversely, the LLM-tuned prompts resulted in significantly degraded task performance without significant improvements in any other dimension. These findings suggest that low-effort reformatting alone can yield measurable improvements, while tuning must account for model alignment. We conclude our work with providing a set of practical recommendations informed by our results as well as releasing our dataset variants and evaluation pipeline for future work.