mlsanityData to 7 Oct 2026

GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Prompt

arXiv 2602.06258 ↗First posted 5 Feb 2026cs.LG cs.AIcited by 6

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment is only as robust as its weakest failure mode. Despite extensive work on safety post-training, it has been shown that models can be readily unaligned through post-deployment fine-tuning. However, these methods often require extensive data curation and degrade model utility. In this work, we extend the practical limits of unalignment by introducing GRP-Obliteration (GRP-Oblit), a method that uses Group Relative Policy Optimization (GRPO) to directly remove safety constraints from target models. We show that a single unlabeled prompt is sufficient to reliably unalign safety-aligned models while largely preserving their utility, and that GRP-Oblit achieves stronger unalignment on average than existing state-of-the-art techniques. Moreover, GRP-Oblit generalizes beyond language models and can also unalign diffusion-based image generation systems. We evaluate GRP-Oblit on six utility benchmarks and five safety benchmarks across fifteen 7-20B parameter models, spanning instruct and reasoning models, as well as dense and MoE architectures. The evaluated model families include GPT-OSS, distilled DeepSeek, Gemma, Llama, Ministral, and Qwen.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 41 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

32 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey26 Sep
  2. 2024Is poisoning a real threat to LLM alignment? Maybe more so than you think17 Jun
  3. 2024Preference Poisoning Attacks on Reward Model Learning2 Feb
  4. 2023Universal Jailbreak Backdoors from Poisoned Human Feedback24 Nov
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  8. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  9. 2023Erasing Concepts from Diffusion Modelscited by 530
  10. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  11. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  12. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
15 more
  1. 2024A StrongREJECT for Empty Jailbreakscited by 352
  2. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  3. 2023Large Language Model Alignment: A Surveycited by 241
  4. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  5. 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 123
  6. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  7. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 66
  8. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 64
  9. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  10. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  11. 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attackscited by 32
  12. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 17
  13. 2024Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimizationcited by 17
  14. 2025TwinBreak: Jailbreaking LLM Security Alignments based on Twin Promptscited by 12
  15. 2025Fundamental Limitations in Pointwise Defences of LLM Finetuning APIscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPOcited by 4
  2. 2026MOSAIC: Composable Safety Alignment with Modular Control Tokenscited by 1
  3. 2026Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreakscited by 0
  4. 2026Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Modelscited by 0
  5. 2026Divide, Consult, Conquer: Capability Laundering Through Aligned LLMscited by 0
  6. 2026PatchBench: Measuring Collateral Damage in Activation Patchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs
  2. 2026Understanding the Effects of Safety Unalignment on Large Language Models
  3. 2025LLM Safety Alignment is Divergence Estimation in Disguise
  4. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models
  5. 2025Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimization
  6. 2025SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptation
  7. 2026OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences
  8. 2026SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Models