# FullStack Bench: Evaluating LLMs as Full Stack Coders

arXiv: https://arxiv.org/abs/2412.00535 · HTML: https://mlsanity.com/paper/2412.00535/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2024.

## Built on by (most cited first, 30 of 58)

- [MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://mlsanity.com/paper/2506.13585/) (2025)
- [WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent](https://mlsanity.com/paper/2508.05748/) (2025)
- [Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey](https://mlsanity.com/paper/2412.20367/) (2024)
- [Dream-Coder 7B: An Open Diffusion Language Model for Code](https://mlsanity.com/paper/2509.01142/) (2025)
- [Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning](https://mlsanity.com/paper/2511.16043/) (2025)
- [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026)
- [Seed-Coder: Let the Code Model Curate Data for Itself](https://mlsanity.com/paper/2506.03524/) (2025)
- [CodeContests+: High-Quality Test Case Generation for Competitive Programming](https://mlsanity.com/paper/2506.05817/) (2025)
- [Teaching Language Models to Critique via Reinforcement Learning](https://mlsanity.com/paper/2502.03492/) (2025)
- [Laminar: A Scalable Asynchronous RL Post-Training Framework](https://mlsanity.com/paper/2510.12633/) (2025)
- [FeatureBench: Benchmarking Agentic Coding for Complex Feature Development](https://mlsanity.com/paper/2602.10975/) (2026)
- [ExecRepoBench: Multi-level Executable Code Completion Evaluation](https://mlsanity.com/paper/2412.11990/) (2024)
- [EpiCoder: Encompassing Diversity and Complexity in Code Generation](https://mlsanity.com/paper/2501.04694/) (2025)
- [SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner](https://mlsanity.com/paper/2506.09003/) (2025)
- [CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation](https://mlsanity.com/paper/2502.19166/) (2025)
- [Generative AI Act II: Test Time Scaling Drives Cognition Engineering](https://mlsanity.com/paper/2504.13828/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models](https://mlsanity.com/paper/2502.16614/) (2025)
- [Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation](https://mlsanity.com/paper/2604.10923/) (2026)
- [Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learning](https://mlsanity.com/paper/2509.22601/) (2025)
- [ERNIE 5.0 Technical Report](https://mlsanity.com/paper/2602.04705/) (2026)
- [SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoning](https://mlsanity.com/paper/2601.04809/) (2026)
- [Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning](https://mlsanity.com/paper/2603.15611/) (2026)
- [Multi-Agent Collaboration for Multilingual Code Instruction Tuning](https://mlsanity.com/paper/2502.07487/) (2025)
- [Efficient Context Scaling with LongCat ZigZag Attention](https://mlsanity.com/paper/2512.23966/) (2025)
- [ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development](https://mlsanity.com/paper/2601.11077/) (2026)
- [IQuest-Coder-V1 Technical Report](https://mlsanity.com/paper/2603.16733/) (2026)
- [IFEvalCode: Controlled Code Generation](https://mlsanity.com/paper/2507.22462/) (2025)
- [InCoder-32B-Thinking: Industrial Code World Model for Thinking](https://mlsanity.com/paper/2604.03144/) (2026)
- [CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks](https://mlsanity.com/paper/2507.05281/) (2025)

## Abstract (arXiv)

As the capabilities of code large language models (LLMs) continue to expand, their applications across diverse code intelligence domains are rapidly increasing. However, most existing datasets only evaluate limited application domains. To address this gap, we have developed a comprehensive code evaluation dataset FullStack Bench focusing on full-stack programming, which encompasses a wide range of application domains (e.g., basic programming, data analysis, software engineering, mathematics, and machine learning). Besides, to assess multilingual programming capabilities, in FullStack Bench, we design real-world instructions and corresponding unit test cases from 16 widely-used programming languages to reflect real-world usage scenarios rather than simple translations. Moreover, we also release an effective code sandbox execution tool (i.e., SandboxFusion) supporting various programming languages and packages to evaluate the performance of our FullStack Bench efficiently. Comprehensive experimental results on our FullStack Bench demonstrate the necessity and effectiveness of our FullStack Bench and SandboxFusion.