# Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks

arXiv: https://arxiv.org/abs/2505.07473 · HTML: https://mlsanity.com/paper/2505.07473/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 12 May 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 19 of 19)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023)
- [CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X](https://mlsanity.com/paper/2303.17568/) (2023)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models](https://mlsanity.com/paper/2503.18971/) (2025)
- [QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks](https://mlsanity.com/paper/2501.17167/) (2025)
- [Planning-Driven Programming: A Large Language Model Programming Workflow](https://mlsanity.com/paper/2411.14503/) (2024)

## Built on by (most cited first, 30 of 40)

- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation](https://mlsanity.com/paper/2507.04952/) (2025)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics](https://mlsanity.com/paper/2601.02430/) (2026)
- [WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning](https://mlsanity.com/paper/2509.22644/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding](https://mlsanity.com/paper/2510.11498/) (2025)
- [HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks](https://mlsanity.com/paper/2604.09937/) (2026)
- [You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluation](https://mlsanity.com/paper/2508.14104/) (2025)
- [IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?](https://mlsanity.com/paper/2509.24709/) (2025)
- [DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle](https://mlsanity.com/paper/2601.20882/) (2026)
- [WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis](https://mlsanity.com/paper/2606.01869/) (2026)
- [MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants](https://mlsanity.com/paper/2603.09652/) (2026)
- [WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics](https://mlsanity.com/paper/2603.13391/) (2026)
- [WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models](https://mlsanity.com/paper/2604.18224/) (2026)
- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments](https://mlsanity.com/paper/2510.26852/) (2025)
- [AppForge: From Assistant to Independent Developer -- Are GPTs Ready for Software Development?](https://mlsanity.com/paper/2510.07740/) (2025)
- [RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation](https://mlsanity.com/paper/2603.25804/) (2026)
- [Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement](https://mlsanity.com/paper/2606.05920/) (2026)
- [Code2Bench: Scaling Source and Rigor for Dynamic Benchmark Construction](https://mlsanity.com/paper/2508.07180/) (2025)
- [WebGen-V Bench: Structured Representation for Enhancing Visual Design in LLM-based Web Generation and Evaluation](https://mlsanity.com/paper/2510.15306/) (2025)
- [Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation](https://mlsanity.com/paper/2605.30000/) (2026)
- [I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications](https://mlsanity.com/paper/2606.00750/) (2026)
- [MIRROR: A Multi-Agent Framework with Iterative Adaptive Revision and Hierarchical Retrieval for Optimization Modeling in Operations Research](https://mlsanity.com/paper/2602.03318/) (2026)
- [FullStack-Agent: Enhancing Agentic Full-Stack Web Coding via Development-Oriented Testing and Repository Back-Translation](https://mlsanity.com/paper/2602.03798/) (2026)
- [PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization](https://mlsanity.com/paper/2605.15222/) (2026)
- [A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistants](https://mlsanity.com/paper/2510.06223/) (2025)
- [Coding with Eyes: Visual Feedback Unlocks Reliable GUI Code Generating and Debugging](https://mlsanity.com/paper/2604.19750/) (2026)
- [Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages](https://mlsanity.com/paper/2606.00154/) (2026)

## Abstract (arXiv)

The application of large language models (LLMs) in the field of coding is evolving rapidly: from code assistants, to autonomous coding agents, and then to generating complete projects through natural language. Early LLM code benchmarks primarily focused on code generation accuracy, but these benchmarks have gradually become saturated. Benchmark saturation weakens their guiding role for LLMs. For example, HumanEval Pass@1 has reached 99.4% and MBPP 94.2%. Among various attempts to address benchmark saturation, approaches based on software engineering have stood out, but the saturation of existing software engineering benchmarks is rapidly increasing. To address this, we propose a new benchmark, Web-Bench, which contains 50 projects, each consisting of 20 tasks with sequential dependencies. The tasks implement project features in sequence, simulating real-world human development workflows. When designing Web-Bench, we aim to cover the foundational elements of Web development: Web Standards and Web Frameworks. Given the scale and complexity of these projects, which were designed by engineers with 5 to 10 years of experience, each presents a significant challenge. On average, a single project takes 4 to 8 hours for a senior engineer to complete. On our given benchmark agent (Web-Agent), SOTA (Claude 3.7 Sonnet) achieves only 25.1% Pass@1, significantly lower (better) than SWE-Bench's Verified (65.4%) and Full (33.8%) scores. Finally, we discuss that in any development field, Standards and Frameworks represent foundational knowledge and efficiency tools, respectively, and LLMs require optimization tailored to them.