mlsanityData to 7 Oct 2026

WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning

arXiv 2312.14187 ↗First posted 20 Dec 2023cs.CL cs.AI cs.SEcited by 42

Not on the map. Why?

Abstract (arXiv)

Recent work demonstrates that, after instruction tuning, Code Large Language Models (Code LLMs) can obtain impressive capabilities to address a wide range of code-related tasks. However, current instruction tuning methods for Code LLMs mainly focus on the traditional code generation task, resulting in poor performance in complex multi-task scenarios. In this paper, we concentrate on multiple code-related tasks and present WaveCoder, a series of Code LLMs trained with Widespread And Versatile Enhanced instruction data. To enable the models to tackle complex code-related tasks, we propose a method to stably generate diverse, high-quality instruction data from open source code dataset in multi-task scenarios and obtain CodeSeaXDataset, a dataset comprising 19,915 instruction instances across 4 code-related tasks, which is aimed at improving the generalization ability of Code LLM. Our experiments demonstrate that WaveCoder models significantly outperform other open-source models in terms of the generalization ability across different code-related tasks. Moreover, WaveCoder-Ultra-6.7B presents the state-of-the-art generalization abilities on a wide range of code-related tasks.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 20 Dec 2023.

Builds on

23 of 28 references.

References · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
11 more
  1. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  2. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  3. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  5. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  6. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  7. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 90
  8. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  9. 2023Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuningcited by 56
  10. 2023Understanding In-Context Learning from Repetitionscited by 27
  11. 2023Instruction Tuned Models are Quick Learnerscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 22; 2026: 5.

42 papers.

Sort:

By area:

  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  2. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  3. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  4. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  5. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  6. 2025FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementationcited by 57
  7. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  8. 2024CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repaircited by 41
  9. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  10. 2025Z1: Efficient Test-time Scaling with Codecited by 35
  11. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  12. 2024LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issuescited by 10
30 more
  1. 2025Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferencescited by 10
  2. 2025GLaD: Geometric Latent Distillation for Vision-Language-Action Modelscited by 10
  3. 2024SRA-MCTS: Self-driven Reasoning Augmentation with Monte Carlo Tree Search for Code Generationcited by 9
  4. 2025AlphaResearch: Accelerating New Algorithm Discovery with Language Modelscited by 9
  5. 2025RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generationcited by 8
  6. 2025Risk Assessment Framework for Code LLMs via Leveraging Internal Statescited by 8
  7. 2024Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiencycited by 7
  8. 2024InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instructcited by 7
  9. 2025The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Modelscited by 7
  10. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  11. 2025Multi-Agent Collaboration for Multilingual Code Instruction Tuningcited by 6
  12. 2024ADC: Enhancing Function Calling Via Adversarial Datasets and Code Line-Level Feedbackcited by 5
  13. 2025CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedbackcited by 4
  14. 2024Towards Better Code Understanding in Decoder-Only Models with Contrastive Learningcited by 3
  15. 2024Mastering the Craft of Data Synthesis for CodeLLMscited by 3
  16. 2025GiFT: Gibbs Fine-Tuning for Code Generationcited by 3
  17. 2025Evaluating the Generalizability of LLMs in Automated Program Repaircited by 3
  18. 2025Is Your Prompt Poisoning Code? Defect Induction Rates and Security Mitigation Strategiescited by 3
  19. 2025Robust Learning of Diverse Code Editscited by 2
  20. 2025CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generationcited by 2
  21. 2025Generating High-Quality Datasets for Code Editing via Open-Source Language Modelscited by 2
  22. 2026Demystifying Data Organization for Enhanced LLM Trainingcited by 2
  23. 2024Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychologycited by 1
  24. 2025Success is in the Details: Evaluate and Enhance Details Sensitivity of Code LLMs through Counterfactualscited by 1
  25. 2025Efficient Code LLM Training via Distribution-Consistent and Diversity-Aware Data Selectioncited by 1
  26. 2026Self-Improving Code Generation via Semantic Entropy and Behavioral Consensuscited by 1
  27. 2026Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Codecited by 1
  28. 2024WarriorCoder: Learning from Expert Battles to Augment Code Large Language Modelscited by 0
  29. 2026X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Testscited by 0
  30. 2026Post-Training Leaves Behavioral Shadows on Unrelated Decisionscited by 0