# LIFEBench: Evaluating Length Instruction Following in Large Language Models

arXiv: https://arxiv.org/abs/2505.16234 · HTML: https://mlsanity.com/paper/2505.16234/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Built on by (most cited first, 12 of 12)

- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale](https://mlsanity.com/paper/2606.15079/) (2026)
- [Parallel Context Compaction for Long-Horizon LLM Agent Serving](https://mlsanity.com/paper/2605.23296/) (2026)
- [On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility](https://mlsanity.com/paper/2605.01357/) (2026)
- [LogicIF: Towards Complex Logic Instruction Following](https://mlsanity.com/paper/2508.09125/) (2025)
- [Plug-and-Play Dramaturge: A Divide-and-Conquer Approach for Iterative Narrative Script Refinement via Collaborative LLM Agents](https://mlsanity.com/paper/2510.05188/) (2025)
- [LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models](https://mlsanity.com/paper/2603.19255/) (2026)
- [TiCo: Time-Controllable Spoken Dialogue Model](https://mlsanity.com/paper/2603.22267/) (2026)
- [Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling](https://mlsanity.com/paper/2604.27039/) (2026)
- [ScopeIF: Improving Scope-Aware Precise Instruction-Following in Large Language Models via Graded Reward Modeling](https://mlsanity.com/paper/2609.32189/) (2026)
- [Small but Mighty: Test Suites as Improvement Signals for Self-Evolving Agents](https://mlsanity.com/paper/2610.15964/) (2026)
- [Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following](https://mlsanity.com/paper/2610.19525/) (2026)

## Abstract (arXiv)

While large language models (LLMs) can solve PhD-level reasoning problems over long context inputs, they still struggle with a seemingly simpler task: following explicit length instructions-e.g., write a 10,000-word novel. Additionally, models often generate far too short outputs, terminate prematurely, or even refuse the request. Existing benchmarks focus primarily on evaluating generations quality, but often overlook whether the generations meet length constraints. To this end, we introduce Length Instruction Following Evaluation Benchmark (LIFEBench) to comprehensively evaluate LLMs' ability to follow length instructions across diverse tasks and a wide range of specified lengths. LIFEBench consists of 10,800 instances across 4 task categories in both English and Chinese, covering length constraints ranging from 16 to 8192 words. We evaluate 26 widely-used LLMs and find that most models reasonably follow short-length instructions but deteriorate sharply beyond a certain threshold. Surprisingly, almost all models fail to reach the vendor-claimed maximum output lengths in practice, as further confirmed by our evaluations extending up to 32K words. Even long-context LLMs, despite their extended input-output windows, counterintuitively fail to improve length-instructions following. Notably, Reasoning LLMs outperform even specialized long-text generation models, achieving state-of-the-art length following. Overall, LIFEBench uncovers fundamental limitations in current LLMs' length instructions following ability, offering critical insights for future progress.