# Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

arXiv: https://arxiv.org/abs/2604.27039 · HTML: https://mlsanity.com/paper/2604.27039/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 36 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 30 of 32)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning](https://mlsanity.com/paper/2504.11456/) (2025)
- [AdaptThink: Reasoning Models Can Learn When to Think](https://mlsanity.com/paper/2505.13417/) (2025)
- [CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling](https://mlsanity.com/paper/1811.10996/) (2018)
- [Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline](https://mlsanity.com/paper/2305.13144/) (2023)
- [Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning](https://mlsanity.com/paper/2506.05256/) (2025)
- [DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning](https://mlsanity.com/paper/2510.15110/) (2025)
- [OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique](https://mlsanity.com/paper/2507.09075/) (2025)
- [Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective](https://mlsanity.com/paper/2506.05754/) (2025)
- [Precise Length Control in Large Language Models](https://mlsanity.com/paper/2412.11937/) (2024)
- [Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model](https://mlsanity.com/paper/2512.21540/) (2025)
- [LIFEBench: Evaluating Length Instruction Following in Large Language Models](https://mlsanity.com/paper/2505.16234/) (2025)
- [Predicting LLM Output Length via Entropy-Guided Representations](https://mlsanity.com/paper/2602.11812/) (2026)
- [Length Controlled Generation for Black-box LLMs](https://mlsanity.com/paper/2412.14656/) (2024)
- [V0: A Generalist Value Model for Any Policy at State Zero](https://mlsanity.com/paper/2602.03584/) (2026)
- [Planning-Aware Code Infilling via Horizon-Length Prediction](https://mlsanity.com/paper/2410.03103/) (2024)
- [Pretrain Value, Not Reward: Decoupled Value Policy Optimization](https://mlsanity.com/paper/2502.16944/) (2025)
- [Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation](https://mlsanity.com/paper/2601.01768/) (2026)

## Abstract (arXiv)

Tokens are the fundamental units of computation in modern autoregressive models, and generation length directly influences both inference cost and reasoning performance. Despite its importance, existing approaches model length primarily at the coarse sequence level. We introduce the Length Value Model (LenVM), a token-level framework that estimates the remaining generation length at every decoding step. By formulating length modeling as a value estimation problem and assigning a constant negative reward to each generated token, LenVM predicts a bounded, discounted return that is a monotone proxy for the remaining generation horizon. This value formulation provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that LenVM supports exact control, continuous performance--efficiency steering, length prediction, and interpretation. On LIFEBench-token, it raises the exact-length score of Qwen2.5-7B-Instruct from 30.9 to 64.8 in one pass and reaches 83.6 when combined with LCG. Near 200 tokens on GSM8K, LenVM retains about 63% Pass@1 versus 6% under a hard token budget. Across other settings, LenVM improves matched-length quality over prompting and EOS calibration, while its length-prediction accuracy improves with scale and exceeds specialized predictors. Its token-level values also identify markers of shifts between longer and shorter trajectories. These results establish generation length as an effective scalable token-level value signal for control, prediction, and future value-based training.