mlsanityData to 7 Oct 2026

Palette: A Modular, Controllable, and Efficient Framework for On-demand Authorized Safety Alignment Relaxation in LLMs

arXiv 2605.24154 ↗First posted 22 May 2026cs.AI cs.SEcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Current safety alignment of foundation models largely follows a one-size-fits-all paradigm, applying the same refusal policy across users and contexts. As a result, models may refuse requests that are unsafe for general users but legitimate for authorized professionals, limiting helpfulness in specialized professional settings. Existing approaches either require costly realignment or rely on inference-time steering that suffers from imprecise control and added latency. To this end, we propose textscPalette, a modular, controllable, and efficient framework that selectively relaxes refusal behavior on authorized target domains while preserving standard safety elsewhere. Our method identifies a refusal direction via multi-objective search and internalizes it into the model through lightweight adaptation. textscPalette further supports modular composition: it learns domain-specific safety controls independently and composes them through parameter merging, enabling on-demand multi-domain authorization without retraining. Experiments across four safety benchmarks, multiple model variants, and both LLMs and VLMs show that textscPalette delivers precise safety control without sacrificing general utility, offering a practical path toward foundation models that adapt to diverse professional needs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 67 of its 72 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

67 of 72 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning6 Jul
  2. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements11 Oct
  3. 2024Emerging Safety Attack and Defense in Federated Instruction Tuning of Large Language Models15 Jun
  4. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  6. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models4 Oct
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 58

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  10. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
38 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  6. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  8. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  9. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  10. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  11. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  12. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 378
  13. 2024A StrongREJECT for Empty Jailbreakscited by 352
  14. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  15. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  16. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  17. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  18. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  19. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  20. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  21. 2023Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimizationcited by 121
  22. 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 105
  23. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  24. 2024Representation Surgery for Multi-Task Model Mergingcited by 95
  25. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  26. 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 72
  27. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  28. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Modelcited by 57
  29. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  30. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  31. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  32. 2025Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineeringcited by 34
  33. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  34. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  35. 2025Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Modelscited by 22
  36. 2025HyperSteer: Activation Steering at Scale with Hypernetworkscited by 20
  37. 2024PaCE: Parsimonious Concept Engineering for Large Language Modelscited by 19
  38. 2025Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steeringcited by 18

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Online Skill Learning for Web Agents via State-Grounded Dynamic Retrievalcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
  2. 2026MOSAIC: Composable Safety Alignment with Modular Control Tokens
  3. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  4. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samples
  5. 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
  6. 2026RAS: Measuring LLM Safety Through Refusal Alignment
  7. 2026Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
  8. 2026Understanding the Effects of Safety Unalignment on Large Language Models