mlsanityData to 7 Oct 2026

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

arXiv 2203.13474 ↗First posted 25 Mar 2022cs.LG cs.CL cs.PLcited by 876

Not on the map. Why?

Cited by 81 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Program synthesis strives to generate a computer program as a solution to a given problem specification, expressed with input-output examples or natural language descriptions. The prevalence of large language models advances the state-of-the-art for program synthesis, though limited training resources and data impede open access to such models. To democratize this, we train and release a family of large language models up to 16.1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER. We show the utility of the trained model by demonstrating that it is competitive with the previous state-of-the-art on zero-shot Python code generation on HumanEval. We further investigate the multi-step paradigm for program synthesis, where a single program is factorized into multiple prompts specifying subproblems. To this end, we construct an open benchmark, Multi-Turn Programming Benchmark (MTPB), consisting of 115 diverse problem sets that are factorized into multi-turn prompts. Our analysis on MTPB shows that the same intent provided to CODEGEN in multi-turn fashion significantly improves program synthesis over that provided as a single turn. We make the training library JAXFORMER and model checkpoints available as open source contribution: https://github.com/salesforce/CodeGen.

Key facts. Cited by 876 papers in the map (data through 7 Oct 2026). Builds on 29 of its 52 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

29 of 52 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  12. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
17 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  8. 2021Measuring Coding Challenge Competence With APPScited by 879
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  10. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  11. 2019SPoC: Search-based Pseudocode to Codecited by 176
  12. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  13. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  14. 2018Mapping Language to Code in Programmatic Contextcited by 104
  15. 2016Neuro-Symbolic Program Synthesiscited by 84
  16. 2020PyMT5: multi-mode translation of natural language and Python code with transformerscited by 80
  17. 2019JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generationcited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 20; 2023: 215; 2024: 314; 2025: 235; 2026: 92.

876 papers.

Sort:

By area:

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  10. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
38 more
  1. 2023Teaching Large Language Models to Self-Debugcited by 963
  2. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  3. 2023StarCoder: may the source be with you!cited by 790
  4. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  6. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  7. 2024A Survey on Large Language Models for Code Generationcited by 646
  8. 2023A Comprehensive Overview of Large Language Modelscited by 639
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2023Textbooks Are All You Needcited by 561
  11. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  12. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  13. 2024Large Language Models: A Surveycited by 506
  14. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  15. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  16. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  17. 2023Scaling Data-Constrained Language Modelscited by 391
  18. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  19. 2022CodeT: Code Generation with Generated Testscited by 369
  20. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
  21. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 346
  22. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  23. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  24. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  25. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  26. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 278
  27. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  28. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
  29. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 242
  30. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  31. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  32. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  33. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  34. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  35. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  36. 2023Efficient Large Language Models: A Surveycited by 183
  37. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  38. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 181

50 most cited of 876 All 876 (JSON)