mlsanityData to 7 Oct 2026

An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS

arXiv 2609.13624 ↗First posted 12 Sep 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large Language Models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences, often producing biased, toxic, or otherwise harmful outputs. Existing alignment methods, while effective, are costly and tightly coupled to the model, limiting flexibility and scalability. We propose a modular correction framework that augments pretrained LLMs with Activated LoRA (aLoRA) adapters and a context-aware routing mechanism to eliminate harms from misaligned model responses. Our approach enables expert adapters to activate mid-sequence without invalidating the KV cache, allowing low-latency, targeted correction during generation. Each expert is trained to detect and mitigate specific harms, such as bias or toxicity. A learned router dynamically selects appropriate experts based on the models intermediate outputs. We demonstrate that our system improves alignment on standard safety benchmarks while preserving task performance, offering a lightweight and efficient path toward safer and more controllable LLM deployments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 21 of its 27 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

21 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  8. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  9. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  10. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  11. 2024MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Expertscited by 144
  12. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
8 more
  1. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  2. 2024X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Designcited by 41
  3. 2024MAP: Multi-Human-Value Alignment Palettecited by 20
  4. 2025KVComm: Enabling Efficient LLM Communication through Selective KV Sharingcited by 16
  5. 2025Activated LoRA: Fine-tuned LLMs for Intrinsicscited by 9
  6. 2025Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRAcited by 7
  7. 2026LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systemscited by 6
  8. 2024Aligners: Decoupling LLMs and Alignmentcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  2. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  3. 2026DNAlign: Dynamic Null-Space Safe Alignment for LLMs
  4. 2026CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment
  5. 2026The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
  6. 2024Locking Down the Finetuned LLMs Safety
  7. 2025Refining Positive and Toxic Samples for Dual Safety Self-Alignment of LLMs with Minimal Human Interventions
  8. 2026Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment