# Beyond Code Generation: LLM-supported Exploration of the Program Design Space

arXiv: https://arxiv.org/abs/2503.06911 · HTML: https://mlsanity.com/paper/2503.06911/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2025.

## Built on by (most cited first, 13 of 13)

- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaboration](https://mlsanity.com/paper/2509.10652/) (2025)
- [Semantic Commit: Helping Users Update Intent Specifications for AI Memory at Scale](https://mlsanity.com/paper/2504.09283/) (2025)
- [Flowco: Rethinking Data Analysis in the Age of LLMs](https://mlsanity.com/paper/2504.14038/) (2025)
- [HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding](https://mlsanity.com/paper/2505.22906/) (2025)
- [From Junior to Senior: Allocating Agency and Navigating Professional Growth in Agentic AI-Mediated Software Engineering](https://mlsanity.com/paper/2602.00496/) (2026)
- [From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach](https://mlsanity.com/paper/2605.21303/) (2026)
- [Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse](https://mlsanity.com/paper/2607.04537/) (2026)
- [Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making](https://mlsanity.com/paper/2602.11924/) (2026)
- [Choose Your Own Adventure: Non-Linear AI-Assisted Programming with EvoGraph](https://mlsanity.com/paper/2604.18883/) (2026)
- [Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agents](https://mlsanity.com/paper/2607.23670/) (2026)
- [Beyond Prompt-to-App: Accountable Translation in Teacher-Facing Agentic Authoring](https://mlsanity.com/paper/2609.04679/) (2026)

## Abstract (arXiv)

In this work, we explore explicit Large Language Model (LLM)-powered support for the iterative design of computer programs. Program design, like other design activity, is characterized by navigating a space of alternative problem formulations and associated solutions in an iterative fashion. LLMs are potentially powerful tools in helping this exploration; however, by default, code-generation LLMs deliver code that represents a particular point solution. This obscures the larger space of possible alternatives, many of which might be preferable to the LLM's default interpretation and its generated code. We contribute an IDE that supports program design through generating and showing new ways to frame problems alongside alternative solutions, tracking design decisions, and identifying implicit decisions made by either the programmer or the LLM. In a user study, we find that with our IDE, users combine and parallelize design phases to explore a broader design space -- but also struggle to keep up with LLM-originated changes to code and other information overload. These findings suggest a core challenge for future IDEs that support program design through higher-level instructions given to LLM-based agents: carefully managing attention and deciding what information agents should surface to program designers and when.