mlsanityData to 7 Oct 2026

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

arXiv 2402.04833 ↗First posted 7 Feb 2024cs.CLcited by 85

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they? LIMA (NeurIPS 2023) and AlpaGasus (ICLR 2024) are state-of-the-art methods for selecting such high-quality examples, either via manual curation or using GPT-3.5-Turbo as a quality scorer. We show that the extremely simple baseline of selecting the 1,000 instructions with longest responses -- that intuitively contain more learnable information and are harder to overfit -- from standard datasets can consistently outperform these sophisticated methods according to GPT-4 and PaLM-2 as judges, while remaining competitive on the Open LLM benchmarks that test factual knowledge. We demonstrate this for several LLMs (Llama-2-7B, Llama-2-13B, Mistral-7B-v0.1) and datasets (Alpaca-52k, Evol-Instruct-70k). In addition, a lightweight refinement of such long instructions can further improve the abilities of the fine-tuned LLMs, and allows us to obtain competitive results on MT-Bench and the 2nd highest-ranked Llama-2-7B-based model on AlpacaEval 2.0, while training on only 1,000 examples and no extra preference data. We also conduct a thorough analysis of our models to ensure that their enhanced performance is not simply due to GPT-4's preference for longer responses. Overall, our findings suggest that fine-tuning on the longest responses should be the default baseline for any work on instruction fine-tuning. We provide our code at https://github.com/tml-epfl/long-is-more-for-alignment.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 54 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

44 of 54 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning25 Dec
  2. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 217 Nov
  3. 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment10 Aug
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  5. 2022Training language models to follow instructions with human feedback4 Mar
  6. 2021A General Language Assistant as a Laboratory for Alignment1 Dec

Other references · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2023Mistral 7Bcited by 3,065
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
26 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  5. 2023Instruction Tuning with GPT-4cited by 676
  6. 2022Large Language Models Can Self-Improvecited by 656
  7. 2024Self-Rewarding Language Modelscited by 654
  8. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  9. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  10. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  11. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  12. 2021Efficient Attentions for Long Document Summarizationcited by 342
  13. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  14. 2020The Pushshift Reddit Datasetcited by 289
  15. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
  16. 2023The False Promise of Imitating Proprietary LLMscited by 254
  17. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  18. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 176
  19. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  20. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  21. 2023NEFTune: Noisy Embeddings Improve Instruction Finetuningcited by 99
  22. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 80
  23. 2023Instruction Mining: Instruction Data Selection for Tuning Large Language Modelscited by 57
  24. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  25. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  26. 2023LIMIT: Less Is More for Instruction Tuning Across Evaluation Paradigmscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 32; 2025: 37; 2026: 16.

85 papers.

Sort:

By area:

  1. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  2. 2024UltraMedical: Building Specialized Generalists in Biomedicinecited by 67
  3. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  4. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 48
  5. 2024Following Length Constraints in Instructionscited by 45
  6. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
  7. 2024Instruction Tuning With Loss Over Instructionscited by 40
  8. 2025Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuningcited by 39
  9. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  10. 2025Large-Scale Data Selection for Instruction Tuningcited by 34
  11. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  12. 2025NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Taskscited by 29
38 more
  1. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  2. 2024Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Modelscited by 26
  3. 2024Improving Data Efficiency via Curating LLM-Driven Rating Systemscited by 26
  4. 2025Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Datacited by 26
  5. 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
  6. 2024Is In-Context Learning Sufficient for Instruction Following in LLMs?cited by 19
  7. 2024Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Modelscited by 17
  8. 2024Better Alignment with Instruction Back-and-Forth Translationcited by 16
  9. 2024LLM Roleplay: Simulating Human-Chatbot Interactioncited by 15
  10. 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Qualitycited by 14
  11. 2024CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitionscited by 13
  12. 2024Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluatorscited by 13
  13. 2025R.I.P.: Better Models by Survival of the Fittest Promptscited by 13
  14. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  15. 2025The Era of Real-World Human Interaction: RL from User Conversationscited by 13
  16. 2024Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixturescited by 12
  17. 2024Chasing Random: Instruction Selection Strategies Fail to Generalizecited by 12
  18. 2024Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspectivecited by 12
  19. 2025T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuningcited by 12
  20. 2025Data Mixing Optimization for Supervised Fine-Tuning of Large Language Modelscited by 12
  21. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 11
  22. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
  23. 2024Enabling Weak LLMs to Judge Response Reliability via Meta Rankingcited by 9
  24. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Onlycited by 9
  25. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 9
  26. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instancecited by 8
  27. 2024LIONs: An Empirically Optimized Approach to Align Language Modelscited by 7
  28. 2025Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoningcited by 7
  29. 2025Nile-Chat: Egyptian Language Models for Arabic and Latin Scriptscited by 7
  30. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  31. 2026GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometrycited by 7
  32. 2024Unlocking Large Language Model's Planning Capabilities with Maximum Diversity Fine-tuningcited by 6
  33. 2024SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Modelscited by 6
  34. 2025SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requestscited by 5
  35. 2024The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuningcited by 4
  36. 2024Multi-Objective Linguistic Control of Large Language Modelscited by 4
  37. 2024Find the Intention of Instruction: Comprehensive Evaluation of Instruction Understanding for Large Language Modelscited by 4
  38. 2025Noiser: Bounded Input Perturbations for Attributing Large Language Modelscited by 4

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  2. 2024NILE: Internal Consistency Alignment in Large Language Models
  3. 2025Aligning Instruction Tuning with Pre-training
  4. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  5. 2023Rethinking the Instruction Quality: LIFT is What You Need
  6. 2024Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning
  7. 2024Does Instruction Tuning Make LLMs More Consistent?
  8. 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks