Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims
Not on the map. Why?
Abstract (arXiv)
Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass. Re-scored under this installed-gap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness. In a 46-cell audit, no cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
18 of 22 references.
References · 18
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
- 2018Model Cards for Model Reportingcited by 1,080
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
- 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
- 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
6 more
- 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 171
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 148
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
- 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
- 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
- 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 96
Built on by
Not cited yet.