mlsanityData to 14 Oct 2026

Platypus: Quick, Cheap, and Powerful Refinement of LLMs

arXiv 2308.07317 ↗First posted 14 Aug 2023cs.CLcited by 150

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Platypus, a family of fine-tuned and merged Large Language Models (LLMs) that achieves the strongest performance and currently stands at first place in HuggingFace's Open LLM Leaderboard as of the release date of this work. In this work we describe (1) our curated dataset Open-Platypus, that is a subset of other open datasets and which textitwe release to the public (2) our process of fine-tuning and merging LoRA modules in order to conserve the strong prior of pretrained LLMs, while bringing specific domain knowledge to the surface (3) our efforts in checking for test data leaks and contamination in the training data, which can inform future research. Specifically, the Platypus family achieves strong performance in quantitative LLM metrics across model sizes, topping the global Open LLM leaderboard while using just a fraction of the fine-tuning data and overall compute that are required for other state-of-the-art fine-tuned LLMs. In particular, a 13B Platypus model can be trained on textita single A100 GPU using 25k questions in 5 hours. This is a testament of the quality of our Open-Platypus dataset, and opens opportunities for more improvements in the field. Project page: https://platypus-llm.github.io

Key facts. Cited by 150 papers in the map (data through 14 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

34 of 51 references.

References · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
22 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  4. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  6. 2023LIMA: Less Is More for Alignmentcited by 1,100
  7. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  9. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  12. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  13. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  14. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  15. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  16. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  17. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  18. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 192
  19. 2023Dataset Distillation: A Comprehensive Reviewcited by 127
  20. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  21. 2023Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Modelscited by 65
  22. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 28; 2024: 77; 2025: 29; 2026: 16.

150 papers.

Sort:

By area:

  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  4. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  5. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  6. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  7. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
  8. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  9. 2024Benchmark Data Contamination of Large Language Models: A Surveycited by 147
  10. 2024Nemotron-4 340B Technical Reportcited by 144
  11. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 117
  12. 2024Knowledge Fusion of Large Language Modelscited by 115
38 more
  1. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  2. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  3. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 100
  4. 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 99
  5. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 99
  6. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  7. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  8. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  9. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  10. 2024AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectoriescited by 56
  11. 2024RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptationcited by 48
  12. 2023HAE-RAE Bench: Evaluation of Korean Knowledge in Language Modelscited by 47
  13. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 46
  14. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoningcited by 45
  15. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 43
  16. 2024Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussionscited by 42
  17. 2023E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendationcited by 39
  18. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  19. 2023Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environmentcited by 37
  20. 2024Mitigating Catastrophic Forgetting in Language Transfer via Model Mergingcited by 36
  21. 2024EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summariescited by 35
  22. 2025A Survey on Data Contamination for Large Language Modelscited by 35
  23. 2023Rethinking the Instruction Quality: LIFT is What You Needcited by 34
  24. 2024"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantizationcited by 34
  25. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  26. 2023Batched Low-Rank Adaptation of Foundation Modelscited by 32
  27. 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarkscited by 32
  28. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  29. 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 30
  30. 2024Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learningcited by 30
  31. 2023How Far Have We Gone in Vulnerability Detection Using Large Language Modelscited by 27
  32. 2024Data Contamination Can Cross Language Barrierscited by 26
  33. 2024Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptionscited by 26
  34. 2024Data Contamination Calibration for Black-box LLMscited by 24
  35. 2025To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoningcited by 24
  36. 2025Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Reportcited by 23
  37. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decodingcited by 22
  38. 2025Sample, Don't Search: Rethinking Test-Time Alignment for Language Modelscited by 22

50 most cited of 150 All 150 (JSON)