mlsanityData to 14 Oct 2026

Orca: Progressive Learning from Complex Explanation Traces of GPT-4

arXiv 2306.02707 ↗First posted 5 Jun 2023cs.CL cs.LGcited by 379

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Recent research has focused on enhancing the capability of smaller models through imitation learning, drawing on the outputs generated by large foundation models (LFMs). A number of issues impact the quality of these models, ranging from limited imitation signals from shallow LFM outputs; small scale homogeneous training data; and most notably a lack of rigorous evaluation resulting in overestimating the small model's capability as they tend to learn to imitate the style, but not the reasoning process of LFMs. To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka.ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs. Orca learns from rich signals from GPT-4 including explanation traces; step-by-step thought processes; and other complex instructions, guided by teacher assistance from ChatGPT. To promote this progressive learning, we tap into large-scale and diverse imitation data with judicious sampling and selection. Orca surpasses conventional state-of-the-art instruction-tuned models such as Vicuna-13B by more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard (BBH) and 42% on AGIEval. Moreover, Orca reaches parity with ChatGPT on the BBH benchmark and shows competitive performance (4 pts gap with optimized system message) in professional and academic examinations like the SAT, LSAT, GRE, and GMAT, both in zero-shot settings without CoT; while trailing behind GPT-4. Our research indicates that learning from step-by-step explanations, whether these are generated by humans or more advanced AI models, is a promising direction to improve model capabilities and skills.

Key facts. Cited by 379 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 32 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

25 of 32 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  12. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
13 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  2. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  4. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  6. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  7. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  8. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  9. 2023The False Promise of Imitating Proprietary LLMscited by 263
  10. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 207
  11. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 184
  12. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 47
  13. 2020XtremeDistil: Multi-stage Distillation for Massive Multilingual Modelscited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 61; 2024: 153; 2025: 78; 2026: 87.

379 papers.

Sort:

By area:

  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  2. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  3. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  4. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  5. 2023Textbooks Are All You Needcited by 569
  6. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  7. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  8. 2024Large Language Models: A Surveycited by 525
  9. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  10. 2023Explainability for Large Language Models: A Surveycited by 333
  11. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  12. 2023Improving Text Embeddings with Large Language Modelscited by 303
38 more
  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  3. 2023Challenges and Applications of Large Language Modelscited by 254
  4. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  5. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  6. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  7. 2025Phi-4-reasoning Technical Reportcited by 151
  8. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  9. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  10. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Datasetcited by 143
  11. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 142
  12. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 134
  13. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 132
  14. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  15. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  16. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  17. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  18. 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 122
  19. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
  20. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  21. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  22. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  23. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 109
  24. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  25. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  26. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  27. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 96
  28. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  29. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  30. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 88
  31. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  32. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  33. 2024InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modelingcited by 79
  34. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 77
  35. 2023Lemur: Harmonizing Natural Language and Code for Language Agentscited by 72
  36. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 70
  37. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  38. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 67

50 most cited of 379 All 379 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning
  2. 2024Diverse capability and scaling of diffusion and auto-regressive models when learning abstract rules
  3. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning
  4. 2024Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought
  5. 2025Intern-S1: A Scientific Multimodal Foundation Model
  6. 2024Overcoming Knowledge Barriers: Online Imitation Learning from Visual Observation with Pretrained World Models
  7. 2025The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Models
  8. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning