mlsanityData to 7 Oct 2026

Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

arXiv 2604.27039 ↗First posted 29 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Tokens are the fundamental units of computation in modern autoregressive models, and generation length directly influences both inference cost and reasoning performance. Despite its importance, existing approaches model length primarily at the coarse sequence level. We introduce the Length Value Model (LenVM), a token-level framework that estimates the remaining generation length at every decoding step. By formulating length modeling as a value estimation problem and assigning a constant negative reward to each generated token, LenVM predicts a bounded, discounted return that is a monotone proxy for the remaining generation horizon. This value formulation provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that LenVM supports exact control, continuous performance--efficiency steering, length prediction, and interpretation. On LIFEBench-token, it raises the exact-length score of Qwen2.5-7B-Instruct from 30.9 to 64.8 in one pass and reaches 83.6 when combined with LCG. Near 200 tokens on GSM8K, LenVM retains about 63% Pass@1 versus 6% under a hard token budget. Across other settings, LenVM improves matched-length quality over prompting and EOS calibration, while its length-prediction accuracy improves with scale and exceeds specialized predictors. Its token-level values also identify markers of shifts between longer and shorter trajectories. These results establish generation length as an effective scalable token-level value signal for control, prediction, and future value-based training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

32 of 36 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
20 more
  1. 2022Efficiently Scaling Transformer Inferencecited by 529
  2. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  3. 2025AdaptThink: Reasoning Models Can Learn When to Thinkcited by 141
  4. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 93
  5. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipelinecited by 86
  6. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learningcited by 51
  7. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningcited by 34
  8. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 24
  9. 2025Constrained Sampling for Language Models Should Be Easy: An MCMC Perspectivecited by 14
  10. 2024Precise Length Control in Large Language Modelscited by 13
  11. 2025Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Modelcited by 13
  12. 2025LIFEBench: Evaluating Length Instruction Following in Large Language Modelscited by 10
  13. 2026Predicting LLM Output Length via Entropy-Guided Representationscited by 9
  14. 2024Length Controlled Generation for Black-box LLMscited by 8
  15. 2026V0: A Generalist Value Model for Any Policy at State Zerocited by 7
  16. 2024Planning-Aware Code Infilling via Horizon-Length Predictioncited by 6
  17. 2025Pretrain Value, Not Reward: Decoupled Value Policy Optimizationcited by 6
  18. 2025Prompt-Based One-Shot Exact Length-Controlled Generation with LLMscited by 4
  19. 2026Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulationcited by 4
  20. 2025Plan-and-Write: Structure-Guided Length Control for LLMs without Model Retrainingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction
  2. 2026When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Prediction
  3. 2025Length-MAX Tokenizer for Language Models
  4. 2025Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models
  5. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
  6. 2026Compute Where it Counts: Self Optimizing Language Models
  7. 2026Test-Time Speculation
  8. 2026Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs