mlsanityData to 7 Oct 2026

FullStack Bench: Evaluating LLMs as Full Stack Coders

arXiv 2412.00535 ↗First posted 30 Nov 2024cs.AI cs.SEcited by 56

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As the capabilities of code large language models (LLMs) continue to expand, their applications across diverse code intelligence domains are rapidly increasing. However, most existing datasets only evaluate limited application domains. To address this gap, we have developed a comprehensive code evaluation dataset FullStack Bench focusing on full-stack programming, which encompasses a wide range of application domains (e.g., basic programming, data analysis, software engineering, mathematics, and machine learning). Besides, to assess multilingual programming capabilities, in FullStack Bench, we design real-world instructions and corresponding unit test cases from 16 widely-used programming languages to reflect real-world usage scenarios rather than simple translations. Moreover, we also release an effective code sandbox execution tool (i.e., SandboxFusion) supporting various programming languages and packages to evaluate the performance of our FullStack Bench efficiently. Comprehensive experimental results on our FullStack Bench demonstrate the necessity and effectiveness of our FullStack Bench and SandboxFusion.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Nov 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 24; 2026: 30.

56 papers.

Sort:

By area:

  1. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  2. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  3. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 82
  4. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  5. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  6. 2026Qwen3-Coder-Next Technical Reportcited by 60
  7. 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 46
  8. 2025CodeContests+: High-Quality Test Case Generation for Competitive Programmingcited by 39
  9. 2025Teaching Language Models to Critique via Reinforcement Learningcited by 38
  10. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 35
  11. 2026FeatureBench: Benchmarking Agentic Coding for Complex Feature Developmentcited by 30
  12. 2025EpiCoder: Encompassing Diversity and Complexity in Code Generationcited by 21
38 more
  1. 2024ExecRepoBench: Multi-level Executable Code Completion Evaluationcited by 19
  2. 2025CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generationcited by 18
  3. 2025SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Mannercited by 18
  4. 2025CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Modelscited by 14
  5. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  6. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  7. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 10
  8. 2026Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillationcited by 10
  9. 2026ERNIE 5.0 Technical Reportcited by 9
  10. 2026SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoningcited by 7
  11. 2025Multi-Agent Collaboration for Multilingual Code Instruction Tuningcited by 6
  12. 2025Efficient Context Scaling with LongCat ZigZag Attentioncited by 6
  13. 2026ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Developmentcited by 6
  14. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 6
  15. 2026IQuest-Coder-V1 Technical Reportcited by 6
  16. 2025IFEvalCode: Controlled Code Generationcited by 5
  17. 2026InCoder-32B-Thinking: Industrial Code World Model for Thinkingcited by 4
  18. 2025CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Taskscited by 3
  19. 2025ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Modelscited by 3
  20. 2026GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Tracescited by 3
  21. 2026Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Rewardcited by 3
  22. 2026Discovery and Reinforcement of Tool-Integrated Reasoning Chains via Rollout Treescited by 2
  23. 2025STEPWISE-CODEX-Bench: Evaluating Complex Multi-Function Comprehension and Fine-Grained Execution Reasoningcited by 1
  24. 2025Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmapcited by 1
  25. 2025TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Frameworkcited by 1
  26. 2025UniCode: Augmenting Evaluation for Code Reasoningcited by 1
  27. 2026AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimizationcited by 1
  28. 2026Reinforcement Learning with Conditional Expectation Rewardcited by 1
  29. 2026ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Modelscited by 1
  30. 2026Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimizationcited by 1
  31. 2026Efficient Agentic Reasoning Through Self-Regulated Simulative Planningcited by 1
  32. 2026Rethinking the Evaluation of Efficiency Methods for Multi-Agent Systemscited by 1
  33. 2026Beyond Text-to-SQL: Can LLMs Really Debug Enterprise ETL SQL?cited by 0
  34. 2026CharTool: Tool-Integrated Visual Reasoning for Chart Understandingcited by 0
  35. 2026SWE-WebDevBench: Evaluating Coding Agent Application Platforms as Virtual Software Agenciescited by 0
  36. 2026Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewardscited by 0
  37. 2026Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learningcited by 0
  38. 2026DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Executioncited by 0

50 most cited of 56 All 56 (JSON)