mlsanityData to 14 Oct 2026

SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models

arXiv 2406.12274 ↗First posted 18 Jun 2024cs.CLcited by 23

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety-aligned language models often exhibit fragile and imbalanced safety mechanisms, increasing the likelihood of generating unsafe content. In addition, incorporating new knowledge through editing techniques to language models can further compromise safety. To address these issues, we propose SafeInfer, a context-adaptive, decoding-time safety alignment strategy for generating safe responses to user queries. SafeInfer comprises two phases: the safety amplification phase, which employs safe demonstration examples to adjust the model's hidden states and increase the likelihood of safer outputs, and the safety-guided decoding phase, which influences token selection based on safety-optimized distributions, ensuring the generated content complies with ethical guidelines. Further, we present HarmEval, a novel benchmark for extensive safety evaluations, designed to address potential misuse scenarios in accordance with the policies of leading AI tech giants.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 29 of its 38 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

29 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  10. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
15 more
  1. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  3. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  4. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  5. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  6. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  7. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  8. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141
  9. 2023Stay on topic with Classifier-Free Guidancecited by 91
  10. 2023Controlled Text Generation via Language Model Arithmeticcited by 51
  11. 2022Critic-Guided Decoding for Controlled Text Generationcited by 45
  12. 2022Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Expertscited by 36
  13. 2023PREADD: Prefix-Adaptive Decoding for Controlled Text Generationcited by 29
  14. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 27
  15. 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 11; 2026: 11.

23 papers.

Sort:

By area:

  1. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 30
  2. 2025Advancing LLM Safe Alignment with Safety Representation Rankingcited by 16
  3. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 14
  4. 2025SafeVid: Toward Safety Aligned Video Large Multimodal Modelscited by 9
  5. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  6. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrievalcited by 7
  7. 2025Speculative Safety-Aware Decodingcited by 7
  8. 2024Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Modelscited by 4
  9. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMscited by 4
  10. 2025MemeSense: An Adaptive In-Context Framework for Social Commonsense Driven Meme Moderationcited by 3
  11. 2026CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignmentcited by 2
  12. 2025ProSocialAlign: Preference Conditioned Test Time Alignment in Language Modelscited by 1
11 more
  1. 2026Position: AI Safety Requires Effective Controllabilitycited by 1
  2. 2025Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Studycited by 0
  3. 2026Tutoring Large Language Models to be Domain-adaptive, Precise, and Safecited by 0
  4. 2026DDOR: Delta Debugging for Explainable Overrefusal Testing and Repaircited by 0
  5. 2026Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Samplingcited by 0
  6. 2026Moving the Safety Barrier: Dynamic Routing Adaptive Alignment Against White-Box Attackscited by 0
  7. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0
  8. 2026RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Modelscited by 0
  9. 2026Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Modelscited by 0
  10. 2026ForeSight: Enhancing Risk Monitoring via Early Safety Signal Distillationcited by 0
  11. 2026Tutoring Large Language Models to be Domain-adaptive, Precise and Safecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
  2. 2025Vulnerability Mitigation for Safety-Aligned Language Models via Debiasing
  3. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
  4. 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signals
  5. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements
  6. 2025SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection
  7. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  8. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging