mlsanityData to 14 Oct 2026

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

arXiv 2306.08568 ↗First posted 14 Jun 2023cs.CL cs.AIcited by 733

Not on the map. Why?

Cited by 102 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

Key facts. Cited by 733 papers in the map (data through 14 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

37 of 49 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2022Training Compute-Optimal Large Language Modelscited by 3,019
25 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  4. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  6. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  7. 2023PaLM 2 Technical Reportcited by 1,162
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  10. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  12. 2023StarCoder: may the source be with you!cited by 799
  13. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  14. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  15. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  16. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  17. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  18. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  19. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  20. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  21. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 125
  22. 2023Learning Performance-Improving Code Editscited by 106
  23. 2022Natural Attack for Pre-trained Models of Codecited by 75
  24. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  25. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 53

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 107; 2024: 323; 2025: 196; 2026: 107.

733 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,344
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  6. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  7. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  8. 2024A Survey on Large Language Models for Code Generationcited by 658
  9. 2023A Comprehensive Overview of Large Language Modelscited by 648
  10. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  12. 2023Textbooks Are All You Needcited by 569
38 more
  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  2. 2023The Falcon Series of Open Language Modelscited by 535
  3. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  4. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  5. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  6. 2024LoRA Learns Less and Forgets Lesscited by 328
  7. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
  8. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  9. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  10. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  11. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
  12. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 269
  13. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  14. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  15. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  16. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
  17. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  18. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  19. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  20. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 223
  21. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 201
  22. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
  23. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  24. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  25. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
  26. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  27. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 145
  28. 2024Nemotron-4 340B Technical Reportcited by 144
  29. 2025SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agentscited by 142
  30. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 140
  31. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Modelscited by 131
  32. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  33. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  34. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  35. 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 122
  36. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 119
  37. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  38. 2023SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHFcited by 111

50 most cited of 733 All 733 (JSON)