# SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models

arXiv: https://arxiv.org/abs/2406.12274 · HTML: https://mlsanity.com/paper/2406.12274/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Safety alignment and jailbreak attacks](https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 29 of 29)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding](https://mlsanity.com/paper/2402.08983/) (2024)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots](https://mlsanity.com/paper/2307.08715/) (2023)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics](https://mlsanity.com/paper/2112.08726/) (2021)
- [Inspecting and Editing Knowledge Representations in Language Models](https://mlsanity.com/paper/2304.00740/) (2023)
- [Stay on topic with Classifier-Free Guidance](https://mlsanity.com/paper/2306.17806/) (2023)
- [Controlled Text Generation via Language Model Arithmetic](https://mlsanity.com/paper/2311.14479/) (2023)
- [Critic-Guided Decoding for Controlled Text Generation](https://mlsanity.com/paper/2212.10938/) (2022)
- [Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts](https://mlsanity.com/paper/2212.10543/) (2022)
- [PREADD: Prefix-Adaptive Decoding for Controlled Text Generation](https://mlsanity.com/paper/2307.03214/) (2023)
- [How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries](https://mlsanity.com/paper/2402.15302/) (2024)
- [Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Models](https://mlsanity.com/paper/2401.10647/) (2024)

## Built on by (most cited first, 23 of 23)

- [Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback](https://mlsanity.com/paper/2503.17682/) (2025)
- [Advancing LLM Safe Alignment with Safety Representation Ranking](https://mlsanity.com/paper/2505.15710/) (2025)
- [On Almost Surely Safe Alignment of Large Language Models at Inference-Time](https://mlsanity.com/paper/2502.01208/) (2025)
- [SafeVid: Toward Safety Aligned Video Large Multimodal Models](https://mlsanity.com/paper/2505.11926/) (2025)
- [Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.03098/) (2025)
- [Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval](https://mlsanity.com/paper/2505.15753/) (2025)
- [Speculative Safety-Aware Decoding](https://mlsanity.com/paper/2508.17739/) (2025)
- [Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models](https://mlsanity.com/paper/2410.12880/) (2024)
- [Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs](https://mlsanity.com/paper/2511.00382/) (2025)
- [MemeSense: An Adaptive In-Context Framework for Social Commonsense Driven Meme Moderation](https://mlsanity.com/paper/2502.11246/) (2025)
- [CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment](https://mlsanity.com/paper/2604.00310/) (2026)
- [ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models](https://mlsanity.com/paper/2512.06515/) (2025)
- [Position: AI Safety Requires Effective Controllability](https://mlsanity.com/paper/2605.27117/) (2026)
- [Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study](https://mlsanity.com/paper/2512.15791/) (2025)
- [Tutoring Large Language Models to be Domain-adaptive, Precise, and Safe](https://mlsanity.com/paper/2602.13860/) (2026)
- [DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair](https://mlsanity.com/paper/2606.03601/) (2026)
- [Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling](https://mlsanity.com/paper/2607.20791/) (2026)
- [Moving the Safety Barrier: Dynamic Routing Adaptive Alignment Against White-Box Attacks](https://mlsanity.com/paper/2608.02674/) (2026)
- [Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models](https://mlsanity.com/paper/2608.14797/) (2026)
- [RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Models](https://mlsanity.com/paper/2609.00790/) (2026)
- [Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models](https://mlsanity.com/paper/2609.04714/) (2026)
- [ForeSight: Enhancing Risk Monitoring via Early Safety Signal Distillation](https://mlsanity.com/paper/2609.13737/) (2026)
- [Tutoring Large Language Models to be Domain-adaptive, Precise and Safe](https://mlsanity.com/paper/2609.23071/) (2026)

## Abstract (arXiv)

Safety-aligned language models often exhibit fragile and imbalanced safety mechanisms, increasing the likelihood of generating unsafe content. In addition, incorporating new knowledge through editing techniques to language models can further compromise safety. To address these issues, we propose SafeInfer, a context-adaptive, decoding-time safety alignment strategy for generating safe responses to user queries. SafeInfer comprises two phases: the safety amplification phase, which employs safe demonstration examples to adjust the model's hidden states and increase the likelihood of safer outputs, and the safety-guided decoding phase, which influences token selection based on safety-optimized distributions, ensuring the generated content complies with ethical guidelines. Further, we present HarmEval, a novel benchmark for extensive safety evaluations, designed to address potential misuse scenarios in accordance with the policies of leading AI tech giants.