From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGEN
Not on the map. Why?
Abstract (arXiv)
Despite the rapid progress of large language models (LLMs), their length-controllable text generation (LCTG) ability remains below expectations, posing a major limitation for practical applications. Existing methods mainly focus on end-to-end training to reinforce adherence to length constraints. However, the lack of decomposition and targeted enhancement of LCTG sub-abilities restricts further progress. To bridge this gap, we conduct a bottom-up decomposition of LCTG sub-abilities with human patterns as reference and perform a detailed error analysis. On this basis, we propose MarkerGen, a simple-yet-effective plug-and-play approach that:(1) mitigates LLM fundamental deficiencies via external tool integration;(2) conducts explicit length modeling with dynamically inserted markers;(3) employs a three-stage generation scheme to better align length constraints while maintaining content quality. Comprehensive experiments demonstrate that MarkerGen significantly improves LCTG across various settings, exhibiting outstanding effectiveness and generalizability.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 19 Feb 2025.
Builds on
19 of 29 references.
References · 19
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
- 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 319
- 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
- 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
- 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
- 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
- 2024Controllable Text Generation for Large Language Models: A Surveycited by 55
- 2024Following Length Constraints in Instructionscited by 45
7 more
- 2024HelloBench: Evaluating Long Text Generation Capabilities of Large Language Modelscited by 42
- 2024Language Models Need Inductive Biases to Count Inductivelycited by 25
- 2023Prompt-Based Length Controlled Generation with Reinforcement Learningcited by 14
- 2024Zero-Shot Strategies for Length-Controllable Summarizationcited by 10
- 2024PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awarenesscited by 9
- 2024Length Controlled Generation for Black-box LLMscited by 8
- 2025LLM-Powered Benchmark Factory: Reliable, Generic, and Efficientcited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
- 2024From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation
- 2025Plan-and-Write: Structure-Guided Length Control for LLMs without Model Retraining
- 2025CIE: Controlling Language Model Text Generations Using Continuous Signals
- 2024Prompt-Based Length Controlled Generation with Multiple Control Types
- 2023Controlled Text Generation via Language Model Arithmetic
- 2025Long-Short Alignment for Effective Long-Context Modeling in LLMs
- 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructions