mlsanityData to 7 Oct 2026

SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution

arXiv 2604.19825 ↗First posted 20 Apr 2026cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

State-of-the-art code generation frameworks rely on mental simulation, where LLMs internally trace execution to verify correctness. We expose a fundamental limitation: the Mental-Reality Gap -- where models hallucinate execution traces and confidently validate buggy code. This gap manifests along two orthogonal dimensions: the Specification Gap (overlooking edge cases during planning) and the Verification Gap (hallucinating correct behavior for flawed code). We propose SolidCoder with a simple principle: don't imagine -- execute. The S.O.L.I.D. architecture addresses both dimensions by forcing edge-case awareness before algorithm design and replacing imagined traces with sandboxed execution using property-based oracles. With GPT-4o, SolidCoder achieves state-of-the-art pass@1 performance: 95.7% on HumanEval (+0.6%p), 77.0% on CodeContests (+4.3%p), and 26.7% on APPS (+3.4%p). Ablation reveals that edge-case awareness provides the largest individual gain, while execution grounding catches categorically different errors that specification improvements cannot address. These gains generalize to RL post-trained models, validating that bridging both gap dimensions is essential for robust code synthesis. We release our code and framework to facilitate future research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

24 of 35 references.

References · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  9. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  10. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2024Qwen2.5-Coder Technical Reportcited by 1,151
12 more
  1. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  2. 2021Measuring Coding Challenge Competence With APPScited by 879
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  4. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  5. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  6. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  7. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  8. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  9. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  10. 2023Large Language Models as Analogical Reasonerscited by 107
  11. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  12. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31

Built on by

Not cited yet.