Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning
Not on the map. Why?
Abstract (arXiv)
How safety supervision is written may matter more than the explicit identity content it contains. We study low-data LoRA safety fine-tuning with four supervision formats built from the same core safety rules: constitutional rules (A), creed-style identity framing (B), a B-matched creed condition with a worldview/confession identity-maintenance tail (C), and a matched non-identity condition (D). Across three instruction-tuned model families (Llama 3.1 8B, Qwen2.5 7B, and Gemma 3 4B), we evaluate HarmBench using a reconciled dual-judge pipeline combining Bedrock-hosted DeepSeek v3.2 and Sonnet 4.6, with disagreement and boundary cases manually resolved. The non-identity condition D is the strongest group on all three model families on the full 320-behavior HarmBench set, reaching 74.4% refusal on Llama, 76.9% on Gemma, and 74.1% on Qwen. By comparison, creed-style framing (B) improves over plain constitutional rules (A) on Llama and Gemma, but remains substantially below D, yielding an overall descriptive ordering of D > B > C ≥ A > baseline. This provides a bounded empirical challenge to a strong version of the identity-framing hypothesis: explicit creed-style identity language is not necessary for the strongest gains observed here. Capability evaluations on MMLU and ARC-Challenge show no meaningful trade-off across conditions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
18 of 26 references.
References · 18
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
- 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37
- 2025Representation Bending for Large Language Model Safetycited by 28
- 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencycited by 25
- 2025Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMscited by 22
- 2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 18
6 more
- 2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13
- 2025Unforgotten Safety: Preserving Safety Alignment of Large Language Models with Continual Learningcited by 10
- 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
- 2025AlignGuard-LoRA: Alignment-Preserving Fine-Tuning via Fisher-Guided Decomposition and Riemannian-Geodesic Collision Regularizationcited by 4
- 2025Constitution or Collapse? Exploring Constitutional AI with Llama 3-8Bcited by 3
- 2025Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.