LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
Not on the map. Why?
Abstract (arXiv)
Large language model serving costs scale directly with output sequence length, yet standard preference alignment often inflates response verbosity without improving utility. We study whether the parameterization of post-training updates affects generation length: low-rank subspaces alter sequence length without modifying the alignment loss. We present LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint. Within this subspace, post-training retains the native preference objective with a frozen backbone. Across Anthropic HH-RLHF dialogue preferences, we evaluate two ~3B decoder backbones, Pythia-2.8B and Qwen2.5-3B, against protocol-matched full-parameter DPO and DrDPO branches and the released SamPO checkpoint. LOCUS reduces continuation length by up to 39.84% on Pythia-2.8B and by 14.87--17.58% on Qwen2.5-3B while updating only 0.24--0.28% of model parameters, with no material change in the internal preference diagnostic.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 10 Sep 2026.
Builds on
22 of 28 references.
References · 22
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2020Learning to summarize from human feedbackcited by 2,562
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
- 2020GLU Variants Improve Transformercited by 1,724
10 more
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2019Root Mean Square Layer Normalizationcited by 1,425
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
- 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
- 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
- 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
- 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
- 2024Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergencecited by 29
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LEAF: Growing Trees Without Branching for Speech-Aware Large Language Model Post-Training
- 2024Impact of Preference Noise on the Alignment Performance of Generative Language Models
- 2022Soft Alignment Objectives for Robust Adaptation of Language Generation
- 2025Sequence-level Large Language Model Training with Contrastive Preference Optimization
- 2025Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization
- 2026Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation
- 2024CORD: Balancing COnsistency and Rank Distillation for Robust Retrieval-Augmented Generation
- 2024TRIM: Token Reduction and Inference Modeling for Cost-Effective Language Generation