Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Education
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Large Language Models (LLMs) are increasingly integrated into educational applications. However, they remain vulnerable to jailbreak and fine-tuning attacks, which can compromise safety alignment and lead to harmful outputs. Existing studies mainly focus on general safety evaluations, with limited attention to the unique safety requirements of educational scenarios. To address this gap, we construct EduHarm, a benchmark containing safe-unsafe instruction pairs across five representative educational scenarios, enabling systematic safety evaluation of educational LLMs. Furthermore, we propose a three-stage shield framework (TSSF) for educational LLMs that simultaneously mitigates both jailbreak and fine-tuning attacks. First, safety-aware attention realignment redirects attention toward critical unsafe tokens, thereby restoring the harmfulness feature that discriminates between unsafe and safe inputs. Second, layer-wise safety judgment identifies harmfulness features by aggregating safety cues across multiple layers to detect unsafe instructions. Finally, defense-driven dual routing separates safe and unsafe queries, ensuring normal processing for benign inputs and guarded responses for harmful ones. Extensive experiments across eight jailbreak attack strategies demonstrate that TSSF effectively strengthens safety while preventing over-refusal of benign queries. Evaluations on three fine-tuning attack datasets further show that it consistently achieves robust defense against harmful queries while maintaining preserving utility gains from benign fine-tuning.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 54 of its 77 references that are in the map. First posted on arXiv 18 Nov 2025.
Builds on
54 of 77 references.
Closest ancestors · 17
Same direction, newest first
- 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation9 Oct
- 2025Attention Slipping: A Mechanistic Understanding of Jailbreak Attacks and Defenses in LLMs6 Jul
- 2024Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models24 Dec
- 2024Tamper-Resistant Safeguards for Open-Weight LLMs1 Aug
- 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis16 Jun
- 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
- 2024Improved Generation of Adversarial Examples Against Safety-aligned LLMs28 May
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
- 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
- 2024AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Models17 Jan
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
5 more
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
- 2023Certifying LLM Safety against Adversarial Prompting6 Sep
- 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 37
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 297
- 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
25 more
- 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
- 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
- 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
- 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 125
- 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
- 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 113
- 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 92
- 2024Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbationcited by 92
- 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
- 2025LLMs Encode Harmfulness and Refusal Separatelycited by 88
- 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 64
- 2024MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Modelscited by 44
- 2024NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuningcited by 37
- 2025Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderationcited by 29
- 2025AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basincited by 19
- 2025False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalizecited by 19
- 2025Poly-Guard: Massive Multi-Domain Safety Policy-Grounded Guardrail Datasetcited by 14
- 2025Secure LLM Fine-Tuning via Safety-Aware Probingcited by 10
- 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 9
- 2025SafeLawBench: Towards Safe Alignment of Large Language Modelscited by 9
- 2025Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraintcited by 7
- 2024Edu-Values: Towards Evaluating the Chinese Education Values of Large Language Modelscited by 5
- 2025Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editingcited by 5
- 2025DSCD: Large Language Model Detoxification with Self-Constrained Decodingcited by 3
- 2024RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaborationcited by 1
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 4
- 2026SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMscited by 3
- 2026SURGENT: A Surgical Multi-Agent Assistance System Across the Perioperative Workflowcited by 3
- 2026Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Resultscited by 2
- 2026Construction-Driven Injection: Linguistically-Grounded Edit-Based Code-Mixing Fingerprints for Large Language Modelscited by 1
- 2026Construction-Driven Injection: Linguistically-Grounded Edit-Based Code-Mixing Fingerprints for Large Language Models28 Jul 2026
- 2026Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Results22 Jul 2026
- 2026SURGENT: A Surgical Multi-Agent Assistance System Across the Perioperative Workflow28 May 2026
- 2026SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs24 Apr 2026
- 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink5 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2024Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models
- 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention
- 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
- 2025Behind the Mask: Benchmarking Camouflaged Jailbreaks in Large Language Models
- 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
- 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
- 2024Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models
- 2025SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models