# Validating AI-Generated Code with Live Programming

arXiv: https://arxiv.org/abs/2306.09541 · HTML: https://mlsanity.com/paper/2306.09541/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2023.

## Built on by (most cited first, 8 of 8)

- [Formal Mathematical Reasoning: A New Frontier in AI](https://mlsanity.com/paper/2412.16075/) (2024)
- [Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agenda](https://mlsanity.com/paper/2507.21928/) (2025)
- [Co-audit: tools to help humans double-check AI-generated content](https://mlsanity.com/paper/2310.01297/) (2023)
- [Human-AI Experience in Integrated Development Environments: A Systematic Literature Review](https://mlsanity.com/paper/2503.06195/) (2025)
- [NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modification](https://mlsanity.com/paper/2508.02823/) (2025)
- [HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding](https://mlsanity.com/paper/2505.22906/) (2025)
- [THE-Tree: Can Tracing Historical Evolution Enhance Scientific Verification and Reasoning?](https://mlsanity.com/paper/2506.21763/) (2025)
- [Beyond Autocomplete: Designing CopilotLens Towards Transparent and Explainable AI Coding Agents](https://mlsanity.com/paper/2506.20062/) (2025)

## Abstract (arXiv)

AI-powered programming assistants are increasingly gaining popularity, with GitHub Copilot alone used by over a million developers worldwide. These tools are far from perfect, however, producing code suggestions that may be incorrect in subtle ways. As a result, developers face a new challenge: validating AI's suggestions. This paper explores whether Live Programming (LP), a continuous display of a program's runtime values, can help address this challenge. To answer this question, we built a Python editor that combines an AI-powered programming assistant with an existing LP environment. Using this environment in a between-subjects study (N=17), we found that by lowering the cost of validation by execution, LP can mitigate over- and under-reliance on AI-generated programs and reduce the cognitive load of validation for certain types of tasks.