mlsanityData to 7 Oct 2026

Magicoder: Empowering Code Generation with OSS-Instruct

arXiv 2312.02120 ↗First posted 4 Dec 2023cs.CL cs.AI cs.SEcited by 251

Not on the map. Why?

Cited by 40 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Magicoder, a series of fully open-source (code, weights, and data) Large Language Models (LLMs) for code that significantly closes the gap with top code models while having no more than 7B parameters. Magicoder models are trained on 75K synthetic instruction data using OSS-Instruct, a novel approach to enlightening LLMs with open-source code snippets to generate diverse instruction data for code. Our main motivation is to mitigate the inherent bias of the synthetic data generated by LLMs through the wealth of open-source references for the production of more realistic and controllable data. The orthogonality of OSS-Instruct and other data generation methods like Evol-Instruct further enables us to build an enhanced MagicoderS. Both Magicoder and MagicoderS substantially outperform state-of-the-art code models with similar or even larger sizes on a wide range of coding benchmarks. Notably, MagicoderS-CL-7B based on CodeLlama even surpasses the prominent ChatGPT on HumanEval+ (66.5 vs. 65.9 in pass@1 ). Overall, OSS-Instruct opens a new direction for crafting diverse synthetic instruction data for code using abundant open-source references.

Key facts. Cited by 251 papers in the map (data through 7 Oct 2026). Builds on 30 of its 50 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

30 of 50 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  9. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  10. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  11. 2021Measuring Coding Challenge Competence With APPScited by 879
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
18 more
  1. 2023Large Language Modelscited by 774
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  3. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  4. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  5. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  6. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  7. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  8. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  9. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  10. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  11. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 181
  12. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  13. 2023Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPTcited by 125
  14. 2023Impact of Code Language Models on Automated Program Repaircited by 85
  15. 2023No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generationcited by 82
  16. 2023Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repaircited by 55
  17. 2017Program Synthesis using Conflict-Driven Learningcited by 26
  18. 2017Program Synthesis using Abstraction Refinementcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 58; 2025: 104; 2026: 89.

251 papers.

Sort:

By area:

  1. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  2. 2025Large Language Diffusion Modelscited by 872
  3. 2025Process Reinforcement through Implicit Rewardscited by 478
  4. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  5. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  6. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  7. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  8. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  9. 2024LoRA vs Full Fine-tuning: An Illusion of Equivalencecited by 102
  10. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  11. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
  12. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
38 more
  1. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  2. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvementcited by 57
  3. 2025FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementationcited by 57
  4. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  5. 2024CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferencescited by 56
  6. 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
  7. 2024Large Language Model for Table Processing: A Surveycited by 52
  8. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  9. 2024A Closer Look into Mixture-of-Experts in Large Language Modelscited by 47
  10. 2025Reasoning to Learn from Latent Thoughtscited by 43
  11. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  12. 2024Meta Large Language Model Compiler: Foundation Models of Compiler Optimizationcited by 39
  13. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  14. 2025MultiKernelBench: A Multi-Platform Benchmark for Kernel Generationcited by 35
  15. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generationcited by 34
  16. 2025MergeBench: A Benchmark for Merging Domain-Specialized LLMscited by 30
  17. 2026SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolvingcited by 30
  18. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 28
  19. 2025Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thoughtcited by 28
  20. 2024SemCoder: Training Code Language Models with Comprehensive Semantics Reasoningcited by 27
  21. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  22. 2024DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graphcited by 26
  23. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  24. 2024PyBench: Evaluating LLM Agent on various real-world coding taskscited by 24
  25. 2025EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Codecited by 24
  26. 2024LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Modelscited by 23
  27. 2025BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillationcited by 23
  28. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  29. 2024Synthesizing Post-Training Data for LLMs through Multi-Agent Simulationcited by 22
  30. 2024Sparsing Law: Towards Large Language Models with Greater Activation Sparsitycited by 22
  31. 2025Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUscited by 22
  32. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  33. 2025OJBench: A Competition Level Code Benchmark For Large Language Modelscited by 22
  34. 2024Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Modelcited by 21
  35. 2025EpiCoder: Encompassing Diversity and Complexity in Code Generationcited by 21
  36. 2024Performance-Aligned LLMs for Generating Fast Codecited by 20
  37. 2026Composer 2 Technical Reportcited by 20
  38. 2024Learning Code Preference via Synthetic Evolutioncited by 19

50 most cited of 251 All 251 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Seed-Coder: Let the Code Model Curate Data for Itself
  2. 2024Case2Code: Scalable Synthetic Data for Code Generation
  3. 2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
  4. 2025Data-efficient LLM Fine-tuning for Code Generation
  5. 2023MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks
  6. 2023Textbooks Are All You Need
  7. 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
  8. 2024Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates