mlsanityData to 7 Oct 2026

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources

arXiv 2306.04751 ↗First posted 7 Jun 2023cs.CLcited by 405

Not on the map. Why?

Cited by 72 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work we explore recent advances in instruction-tuning language models on a range of open instruction-following datasets. Despite recent claims that open models can be on par with state-of-the-art proprietary models, these claims are often accompanied by limited evaluation, making it difficult to compare models across the board and determine the utility of various resources. We provide a large set of instruction-tuned models from 6.7B to 65B parameters in size, trained on 12 instruction datasets ranging from manually curated (e.g., OpenAssistant) to synthetic and distilled (e.g., Alpaca) and systematically evaluate them on their factual knowledge, reasoning, multilinguality, coding, and open-ended instruction following abilities through a collection of automatic, model-based, and human-based metrics. We further introduce T"ulu, our best performing instruction-tuned model suite finetuned on a combination of high-quality open resources. Our experiments show that different instruction-tuning datasets can uncover or enhance specific skills, while no single dataset (or combination) provides the best performance across all evaluations. Interestingly, we find that model and human preference-based evaluations fail to reflect differences in model capabilities exposed by benchmark-based evaluations, suggesting the need for the type of systemic evaluation performed in this work. Our evaluations show that the best model in any given evaluation reaches on average 87% of ChatGPT performance, and 73% of GPT-4 performance, suggesting that further investment in building better base models and instruction-tuning data is required to close the gap. We release our instruction-tuned models, including a fully finetuned 65B T"ulu, along with our code, data, and evaluation framework at https://github.com/allenai/open-instruct to facilitate future research.

Key facts. Cited by 405 papers in the map (data through 7 Oct 2026). Builds on 48 of its 81 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

48 of 81 references.

References · 48

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  11. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
36 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  10. 2023LIMA: Less Is More for Alignmentcited by 1,080
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  12. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  13. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  14. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  15. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  16. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  17. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  18. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  19. 2023Instruction Tuning with GPT-4cited by 676
  20. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  21. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  22. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  23. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  24. 2023Large Language Models as Tool Makerscited by 293
  25. 2023The False Promise of Imitating Proprietary LLMscited by 254
  26. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  27. 2020Learning from Task Descriptionscited by 78
  28. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  29. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  30. 2020Automated Lay Language Summarization of Biomedical Scientific Reviewscited by 46
  31. 2022HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisationcited by 28
  32. 2018Toward Fast and Accurate Neural Discourse Segmentationcited by 19
  33. 2018Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verificationcited by 17
  34. 2020LiveQA: A Question Answering Dataset over Sports Livecited by 15
  35. 2018Bag-of-Words as Target for Neural Machine Translationcited by 11
  36. 2018Tag-Enhanced Tree-Structured Neural Networks for Implicit Discourse Relation Classificationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 84; 2024: 210; 2025: 84; 2026: 27.

405 papers.

Sort:

By area:

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  6. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  9. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  10. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  11. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  12. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
38 more
  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  2. 2024Large Language Models: A Surveycited by 506
  3. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  4. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 365
  5. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  7. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  8. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  9. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  10. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  11. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  12. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  13. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  14. 2024Generative Representational Instruction Tuningcited by 227
  15. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  16. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  17. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  18. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 198
  19. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  20. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  21. 2024A Roadmap to Pluralistic Alignmentcited by 175
  22. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  23. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  24. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  25. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 141
  26. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  27. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  28. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  29. 2023Large language models can accurately predict searcher preferencescited by 129
  30. 2024Personalized Language Modeling from Personalized Human Feedbackcited by 124
  31. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  32. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 123
  33. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 118
  34. 2023PRD: Peer Rank and Discussion Improve Large Language Model based Evaluationscited by 114
  35. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
  36. 2024Knowledge Fusion of Large Language Modelscited by 110
  37. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  38. 2023SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHFcited by 107

50 most cited of 405 All 405 (JSON)