mlsanityData to 7 Oct 2026

WizardLM: Empowering large pre-trained language models to follow complex instructions

arXiv 2304.12244 ↗First posted 24 Apr 2023cs.CL cs.AIcited by 1,116

Not on the map. Why?

Cited by 147 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Human evaluations on a complexity-balanced test bed and Vicuna's testset show that instructions from Evol-Instruct are superior to human-created ones. By analyzing the human evaluation results of the high complexity part, we demonstrate that outputs from our WizardLM are preferred to outputs from OpenAI ChatGPT. In GPT-4 automatic evaluation, WizardLM achieves more than 90% capacity of ChatGPT on 17 out of 29 skills. Even though WizardLM still lags behind ChatGPT in some aspects, our findings suggest that fine-tuning with AI-evolved instructions is a promising direction for enhancing LLMs. Our code and data are public at https://github.com/nlpxucan/WizardLM

Key facts. Cited by 1,116 papers in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 24 Apr 2023.

Builds on

28 of 53 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
16 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  6. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  7. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  8. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  9. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  10. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  11. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  12. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  13. 2023Fairness-guided Few-shot Prompting for Large Language Modelscited by 53
  14. 2023Phoenix: Democratizing ChatGPT across Languagescited by 32
  15. 2023An Evaluation on Large Language Model Outputs: Discourse and Memorizationcited by 20
  16. 2023Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Needcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 207; 2024: 492; 2025: 262; 2026: 155.

1,116 papers.

Sort:

By area:

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  6. 2023Large Language Models as Optimizerscited by 830
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  8. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  9. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  10. 2024Yi: Open Foundation Models by 01.AIcited by 706
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  12. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
38 more
  1. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  2. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  3. 2025Kimi K2: Open Agentic Intelligencecited by 523
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  5. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  6. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  7. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  8. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  9. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  10. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  12. 2023Scaling Data-Constrained Language Modelscited by 391
  13. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  14. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  15. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  16. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  17. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  18. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  19. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  20. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  21. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  22. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  23. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  24. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  25. 2024RLHF Workflow: From Reward Modeling to Online RLHFcited by 265
  26. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  27. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 258
  28. 2023Challenges and Applications of Large Language Modelscited by 251
  29. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  30. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  31. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
  32. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  33. 2023Large Language Model Alignment: A Surveycited by 241
  34. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  35. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 239
  36. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  37. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  38. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222

50 most cited of 1,116 All 1,116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MAmmoTH2: Scaling Instructions from the Web
  2. 2023Instruction Tuning with GPT-4
  3. 2024Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese
  4. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
  5. 2025Aligning Instruction Tuning with Pre-training
  6. 2023An Empirical Study of Instruction-tuning Large Language Models in Chinese
  7. 2025InstructLR: A Scalable Approach to Create Instruction Dataset for Under-Resourced Languages
  8. 2025REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback