mlsanityData to 14 Oct 2026

Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing

arXiv 2603.15647 ↗First posted 4 Mar 2026cs.LG cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Large language models (LLMs) are typically governed by post-training alignment (e.g., RLHF or DPO), which yields a largely static policy during deployment and inference. However, real-world safety is a full-lifecycle problem: static defenses degrade against evolving jailbreak behaviors, and fixed weights cannot adapt to pluralistic, time-varying safety norms. This motivates inference-time governance that steers behavior without costly retraining. To address this, we introduce the Consensus Clustering LinUCB Bandit (CCLUB), a unified framework for adaptive social alignment via system-prompt routing. CCLUB employs a conservative consensus clustering mechanism: it pools data only within the intersection of utility and safety similarity graphs, effectively preventing unsafe generalization across semantically proximal but risk-divergent contexts. Our theoretical analysis yields a sublinear regret guarantee, demonstrating near-optimal performance of CCLUB. Extensive experiments validate that CCLUB outperforms strong baselines, achieving a 10.98% improvement in cumulative reward and a 14.42% reduction in the average suboptimality gap.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 56 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

41 of 56 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models9 Jun
  2. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety26 May
  3. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization17 Apr
  4. 2025Improving LLM Safety Alignment with Dual-Objective Optimization5 Mar
  5. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence24 Feb
  6. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements11 Oct
  7. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  8. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  9. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 31

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2023Steering Language Models With Activation Engineeringcited by 922
  9. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  10. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  11. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  12. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
19 more
  1. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 214
  2. 2025Qwen3Guard Technical Reportcited by 155
  3. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 141
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  5. 2014Online Clustering of Banditscited by 102
  6. 2025Universal Model Routing for Efficient LLM Inferencecited by 86
  7. 2024Cost-Effective Online Multi-LLM Selection with Versatile Reward Modelscited by 39
  8. 2019Improved Algorithm on Online Clustering of Banditscited by 38
  9. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 27
  10. 2017Online Clustering of Contextual Cascading Banditscited by 24
  11. 2025Adaptive LLM Routing under Budget Constraintscited by 24
  12. 2025Hyperbolic Safety-Aware Vision-Language Modelscited by 17
  13. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 14
  14. 2025LLM Routing with Dueling Feedbackcited by 7
  15. 2025Sysformer: Safeguarding Frozen Large Language Models with Adaptive System Promptscited by 5
  16. 2025Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Expertscited by 5
  17. 2024Safe-Embed: Unveiling the Safety-Critical Knowledge of Sentence Encoderscited by 3
  18. 2025Demystifying Online Clustering of Bandits: Enhanced Exploration Under Stochastic and Smoothed Adversarial Contextscited by 3
  19. 2025Leveraging the Power of Conversations: Optimal Key Term Selection in Conversational Contextual Banditscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
  2. 2024Sample-Efficient Alignment for LLMs
  3. 2025Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energy
  4. 2026On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment
  5. 2026Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models
  6. 2026LLM Active Alignment: A Nash Equilibrium Perspective
  7. 2026Cat-DPO: Category-Adaptive Safety Alignment
  8. 2026SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment