mlsanityData to 7 Oct 2026

Boundary Suppression Asymmetry in Post-trained Assistants: Over-expansion as a Controllability Cost

arXiv 2605.27969 ↗First posted 27 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-trained language-model assistants are often optimized to avoid under-answering, encouraging complete, helpful, cautious, and proactive responses. We ask whether this optimization creates asymmetric controllability costs: when users explicitly request narrower answers, which assistant behaviors remain suppressible, and which continue to shape the response? We study this problem as boundary-suppression asymmetry. Prompt-side probes across multiple high-level response dimensions suggest a selective cost, concentrated around `too-much assistant' directions such as over-completion, extra help, and anti-underanswering. Using controlled assistant-policy variants derived from a shared base model, we find that anti-underanswering policies are harder to pull back than the baseline under matched boundary-control evaluations, while minimal-boundary variants generally avoid this anti-side upward shift in the direct boundary-control comparisons. Mechanism-oriented probes point beyond longer default outputs, pure EOS failure, uncertainty compensation, and local continuation bias, while robustness checks preserve the main anti-over-baseline ordering under shared-system and larger-scale settings. The evidence supports a mixed planning/stopping account, where content-budget overshoot and continuation persistence jointly make boundary correction harder. Overall, post-training may create direction-specific controllability costs: some helpful assistant tendencies remain easy to invoke, yet harder to locally suppress.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

15 of 21 references.

References · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2023LIMA: Less Is More for Alignmentcited by 1,080
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  7. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  8. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  9. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  10. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  11. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  12. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
3 more
  1. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  2. 2024Following Length Constraints in Instructionscited by 45
  3. 2024Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspectivecited by 12

Built on by

Not cited yet.