mlsanityData to 7 Oct 2026

OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement

arXiv 2402.14658 ↗First posted 22 Feb 2024cs.SE cs.AI cs.CLcited by 242

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

The introduction of large language models has significantly advanced code generation. However, open-source models often lack the execution capabilities and iterative refinement of advanced systems like the GPT-4 Code Interpreter. To address this, we introduce OpenCodeInterpreter, a family of open-source code systems designed for generating, executing, and iteratively refining code. Supported by Code-Feedback, a dataset featuring 68K multi-turn interactions, OpenCodeInterpreter integrates execution and human feedback for dynamic code refinement. Our comprehensive evaluation of OpenCodeInterpreter across key benchmarks such as HumanEval, MBPP, and their enhanced versions from EvalPlus reveals its exceptional performance. Notably, OpenCodeInterpreter-33B achieves an accuracy of 83.2 (76.4) on the average (and plus versions) of HumanEval and MBPP, closely rivaling GPT-4's 84.2 (76.2) and further elevates to 91.6 (84.6) with synthesized human feedback from GPT-4. OpenCodeInterpreter brings the gap between open-source code generation models and proprietary systems like GPT-4 Code Interpreter.

Key facts. Cited by 242 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 45 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

28 of 45 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback2 Feb
  2. 2023Teaching Large Language Models to Self-Debug11 Apr
  3. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2023Mistral 7Bcited by 3,065
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
13 more
  1. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  4. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  5. 2022Galactica: A Large Language Model for Sciencecited by 736
  6. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  9. 2022Self-critiquing models for assisting human evaluatorscited by 372
  10. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  11. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  12. 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 47
  13. 2021Interscript: A dataset for interactive learning of scripts through error feedbackcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 69; 2025: 96; 2026: 77.

242 papers.

Sort:

By area:

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  4. 2024A Survey on Large Language Models for Code Generationcited by 646
  5. 2025Process Reinforcement through Implicit Rewardscited by 478
  6. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  7. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  8. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  9. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  10. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  11. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  12. 2024MAmmoTH2: Scaling Instructions from the Webcited by 143
38 more
  1. 2024Nemotron-4 340B Technical Reportcited by 143
  2. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  3. 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 100
  4. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  5. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agentscited by 87
  6. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  7. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
  8. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  9. 2024LoRA-Pro: Are Low-Rank Adapters Properly Optimized?cited by 70
  10. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  11. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  12. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  13. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  14. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 61
  15. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  16. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  17. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  18. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 51
  19. 2025FineVision: Open Data Is All You Needcited by 51
  20. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  21. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  22. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 45
  23. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  24. 2024CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuningcited by 41
  25. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  26. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 39
  27. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  28. 2024Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLMcited by 38
  29. 2024CodeV: Empowering LLMs with HDL Generation through Multi-Level Summarizationcited by 38
  30. 2025Vision as LoRAcited by 37
  31. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  32. 2024LAMBDA: A Large Model Based Data Agentcited by 30
  33. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 28
  34. 2024RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projectscited by 25
  35. 2024PyBench: Evaluating LLM Agent on various real-world coding taskscited by 24
  36. 2024Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratchcited by 24
  37. 2024ProSec: Fortifying Code LLMs with Proactive Security Alignmentcited by 24
  38. 2026TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agentscited by 24

50 most cited of 242 All 242 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024An Empirical Study on the Code Refactoring Capability of Large Language Models
  2. 2025BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution
  3. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving
  4. 2025PACIFIC: a framework for generating benchmarks to check Precise Automatically Checked Instruction Following In Code
  5. 2026Review Beats Planning: Dual-Model Interaction Patterns for Code Synthesis ↗
  6. 2024A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement
  7. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique
  8. 2026Vision-Guided Iterative Refinement for Frontend Code Generation