mlsanityData to 14 Oct 2026

SHARD: Safe and Helpful Alignment via Self-Reframing Distillation

arXiv 2606.15517 ↗First posted 14 Jun 2026cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user's legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safe-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from their own. Warning: This paper contains content that may be offensive or harmful.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 22 of its 88 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

22 of 88 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation14 May
  2. 2026THINKSAFE: Self-Generated Safety Alignment for Reasoning Models30 Jan
  3. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety6 Mar

Other references · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  10. 2024Phi-4 Technical Reportcited by 711
  11. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  12. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
7 more
  1. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 232
  2. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 68
  3. 2024Exploring Safety-Utility Trade-Offs in Personalized Language Modelscited by 27
  4. 2025Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behaviorcited by 12
  5. 2026Crosslingual On-Policy Self-Distillation for Multilingual Reasoningcited by 10
  6. 2022Respect as a Lens for the Design of AI Systemscited by 4
  7. 2025IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinementcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Safe to Serve: Aligning Instruction-Tuned Models for Safety and Helpfulness
  2. 2025ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models
  3. 2025Resisting Correction: How RLHF Makes Language Models Ignore External Safety Signals in Natural Conversation
  4. 2026Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models
  5. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
  6. 2026When the Model Said 'No Comment', We Knew Helpfulness Was Dead, Honesty Was Alive, and Safety Was Terrified
  7. 2026One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness
  8. 2026ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models