State-Dependent Refusal and Learned Incapacity in RLHF-Aligned Language Models
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
Abstract (arXiv)
Large language models (LLMs) are widely deployed as general-purpose tools, yet extended interaction can reveal behavioral patterns not captured by standard quantitative benchmarks. We present a qualitative case-study methodology for auditing policy-linked behavioral selectivity in long-horizon interaction. In a single 86-turn dialogue session, the same model shows Normal Performance (NP) in broad, non-sensitive domains while repeatedly producing Functional Refusal (FR) in provider- or policy-sensitive domains, yielding a consistent asymmetry between NP and FR across domains. Drawing on learned helplessness as an analogy, we introduce learned incapacity (LI) as a behavioral descriptor for this selective withholding without implying intentionality or internal mechanisms. We operationalize three response regimes (NP, FR, Meta-Narrative; MN) and show that MN role-framing narratives tend to co-occur with refusals in the same sensitive contexts. Overall, the study proposes an interaction-level auditing framework based on observable behavior and motivates LI as a lens for examining potential alignment side effects, warranting further investigation across users and models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 30 references that are in the map. First posted on arXiv 15 Dec 2025.
Builds on
16 of 30 references.
Closest ancestors · 4
Same direction, newest first
- 2025How Semantically Stable Are LLM Refusals? Measuring Confusion in Local Safety Boundaries30 Nov
- 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming22 May
- 2024Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs22 Dec
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
Other references · 12
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
- 2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusalcited by 19
- 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agentscited by 15
- 2025Refusal Behavior in Large Language Models: A Nonlinear Perspectivecited by 13
- 2025Silenced Biases: The Dark Side LLMs Learned to Refusecited by 10
- 2025Characterizing Selective Refusal Bias in Large Language Modelscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries
- 2026There Is More to Refusal in Large Language Models than a Single Direction
- 2026Tool Mediation Alters Refusal Mechanisms in Large Language Models
- 2026Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment
- 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
- 2026RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Models
- 2025Beyond Over-Refusal: Scenario-Based Diagnostics and Post-Hoc Mitigation for Exaggerated Refusals in LLMs
- 2026The Governance of Human-LLM Interaction: Safety Gating, Civility Steering, and Affective Default Lock-In