mlsanityData to 7 Oct 2026

Following Length Constraints in Instructions

arXiv 2406.17744 ↗First posted 25 Jun 2024cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

Aligned instruction following models can better fulfill user requests than their unaligned counterparts. However, it has been shown that there is a length bias in evaluation of such models, and that training algorithms tend to exploit this bias by learning longer responses. In this work we show how to train models that can be controlled at inference time with instructions containing desired length constraints. Such models are superior in length instructed evaluations, outperforming standard instruction following models such as GPT4, Llama 3 and Mixtral.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

18 of 26 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  8. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  9. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  11. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  12. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
6 more
  1. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  2. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  3. 2017Controllable Abstractive Summarizationcited by 110
  4. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  5. 2023Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedbackcited by 78
  6. 2023Prompt-Based Length Controlled Generation with Reinforcement Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 24; 2026: 10.

45 papers.

Sort:

By area:

  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  2. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  3. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
  4. 2024Thinking LLMs: General Instruction Following with Thought Generationcited by 45
  5. 2025A Survey of Direct Preference Optimizationcited by 35
  6. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  7. 2025Scalable Chain of Thoughts via Elastic Reasoningcited by 32
  8. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  9. 2025From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibriumcited by 24
  10. 2025Train Long, Think Short: Curriculum Learning for Efficient Reasoningcited by 20
  11. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  12. 2025An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraintcited by 16
33 more
  1. 2025R.I.P.: Better Models by Survival of the Fittest Promptscited by 13
  2. 2025Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcementcited by 13
  3. 2025On Reasoning Strength Planning in Large Reasoning Modelscited by 13
  4. 2025The Era of Real-World Human Interaction: RL from User Conversationscited by 13
  5. 2025Order Matters: Investigate the Position Bias in Multi-constraint Instruction Followingcited by 12
  6. 2024Zero-Shot Strategies for Length-Controllable Summarizationcited by 10
  7. 2024Multi-Preference Optimization: Generalizing DPO via Set-Level Contrastscited by 9
  8. 2024Length Desensitization in Direct Preference Optimizationcited by 8
  9. 2024Length Controlled Generation for Black-box LLMscited by 8
  10. 2025Med-U1: Incentivizing Unified Medical Reasoning in LLMs via Large-scale Reinforcement Learningcited by 8
  11. 2025mFollowIR: a Multilingual Benchmark for Instruction Following in Retrievalcited by 7
  12. 2025Disentangling Length Bias In Preference Learning Via Response-Conditioned Modelingcited by 6
  13. 2025Revisiting the Reliability of Language Models in Instruction-Followingcited by 6
  14. 2025AMPO: Active Multi-Preference Optimization for Self-play Preference Selectioncited by 5
  15. 2025MedReadCtrl: Personalizing medical text generation with readability-controlled instruction learningcited by 5
  16. 2026Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributionscited by 5
  17. 2026Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulationcited by 4
  18. 2024Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarizationcited by 3
  19. 2024Self-controller: Controlling LLMs with Multi-round Step-by-step Self-awarenesscited by 2
  20. 2025From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGENcited by 2
  21. 2025One Goal, Many Challenges: Robust Preference Optimization Amid Content-Aware and Multi-Source Noisecited by 2
  22. 2025Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markerscited by 2
  23. 2025Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMscited by 2
  24. 2026Neuro-Symbolic Verification on Instruction Following of LLMscited by 2
  25. 2024Exploring Iterative Controllable Summarization with Large Language Modelscited by 1
  26. 2025CIE: Controlling Language Model Text Generations Using Continuous Signalscited by 1
  27. 2026LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Modelscited by 1
  28. 2026One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulnesscited by 1
  29. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Followingcited by 1
  30. 2026Toward IIT-Inspired Consciousness in LLMs: A Reward-Based Learning Frameworkcited by 0
  31. 2026HotelQuEST: Balancing Quality and Efficiency in Agentic Searchcited by 0
  32. 2026Boundary Suppression Asymmetry in Post-trained Assistants: Over-expansion as a Controllability Costcited by 0
  33. 2026Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulatorscited by 0