# TeleGen: Improving LLM-Based Web Application Generation via Runtime Telemetry

arXiv: https://arxiv.org/abs/2610.04981 · HTML: https://mlsanity.com/paper/2610.04981/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 71 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Builds on (most cited first, 27 of 27)

- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering](https://mlsanity.com/paper/2403.03163/) (2024)
- [WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch](https://mlsanity.com/paper/2505.03733/) (2025)
- [Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping](https://mlsanity.com/paper/2411.03292/) (2024)
- [Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks](https://mlsanity.com/paper/2505.07473/) (2025)
- [WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollback](https://mlsanity.com/paper/2505.20013/) (2025)
- [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://mlsanity.com/paper/2509.22644/) (2025)
- [UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization](https://mlsanity.com/paper/2511.08195/) (2025)
- [SemAgent: A Semantics Aware Program Repair Agent](https://mlsanity.com/paper/2506.16650/) (2025)
- [Studying Duplicate Logging Statements and Their Relationships with Code Clones](https://mlsanity.com/paper/2106.00339/) (2021)
- [Are Autonomous Web Agents Good Testers?](https://mlsanity.com/paper/2504.01495/) (2025)
- [MM-WebAgent: A Hierarchical Multimodal Web Agent for Webpage Generation](https://mlsanity.com/paper/2604.15309/) (2026)
- [Envisioning Future Interactive Web Development: Editing Webpage with Natural Language](https://mlsanity.com/paper/2510.26516/) (2025)

## Abstract (arXiv)

Large language models can generate runnable web applications from natural-language requirements, but many generated applications still fail interactive tasks. Existing generate-execute-repair pipelines execute the generated application and use task outcomes or error messages to guide code revision. However, this feedback often misses the runtime behavior between a browser action and the final task outcome, making interaction-level failures difficult to diagnose. Therefore, we propose TeleGen, an observability-enhanced framework for LLM-based web application generation. TeleGen instruments generated applications, collects runtime telemetry during task execution, and compresses raw telemetry logs into concise briefs for repair. We evaluate TeleGen on WebGen-Bench and Web-Bench. On WebGen-Bench, TeleGen improves task success from 67.7% with repair without telemetry to 76.2%, an increase of 8.5 percentage points. On Web-Bench, it improves cumulative Pass@2 from 21.7% to 29.8%. Ablation results show that runtime telemetry provides a useful diagnostic signal, while telemetry briefs make this signal more effective and less costly to use. Further analysis shows that telemetry is especially helpful for failures involving hidden execution paths, such as navigation, form workflows, and frontend-backend coordination.