mlsanityData to 7 Oct 2026

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance

arXiv 2401.11206 ↗First posted 20 Jan 2024cs.CLcited by 78

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the requirements of different applications. A pivotal factor in the success of current LLMs is the alignment process. Current alignment methods, such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), focus on training-time alignment and are often complex and cumbersome to implement. Therefore, we develop InferAligner, a novel inference-time alignment method that utilizes cross-model guidance for harmlessness alignment. InferAligner utilizes safety steering vectors extracted from safety-aligned model to modify the activations of the target model when responding to harmful inputs, thereby guiding the target model to provide harmless responses. Experimental results show that our method can be very effectively applied to domain-specific models in finance, medicine, and mathematics, as well as to multimodal large language models (MLLMs) such as LLaVA. It significantly diminishes the Attack Success Rate (ASR) of both harmful instructions and jailbreak attacks, while maintaining almost unchanged performance in downstream tasks.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 45 references that are in the map. First posted on arXiv 20 Jan 2024.

Builds on

32 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2023Qwen Technical Reportcited by 3,273
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
18 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2023LIMA: Less Is More for Alignmentcited by 1,080
  3. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  4. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  5. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  6. 2023Steering Language Models With Activation Engineeringcited by 824
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  8. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  9. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  10. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  11. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  12. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  13. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  14. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  15. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  16. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  17. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  18. 2023SeqXGPT: Sentence-Level AI-Generated Text Detectioncited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 28; 2026: 24.

78 papers.

Sort:

By area:

  1. 2025MemOS: A Memory OS for AI Systemcited by 129
  2. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  3. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  4. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  5. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  6. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  7. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  8. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  9. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
  10. 2025Safety in Large Reasoning Models: A Surveycited by 58
  11. 2024SpeechAlign: Aligning Speech Generation to Human Preferencescited by 45
  12. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
38 more
  1. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 39
  2. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  3. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  4. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  5. 2024Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methodscited by 26
  6. 2024Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Modelcited by 24
  7. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Modelscited by 24
  8. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Modelscited by 23
  9. 2024Locking Down the Finetuned LLMs Safetycited by 22
  10. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  11. 2024Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Modelscited by 18
  12. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  13. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  14. 2024Towards Inference-time Category-wise Safety Steering for Large Language Modelscited by 17
  15. 2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
  16. 2025Automating Steering for Safe Multimodal Large Language Modelscited by 17
  17. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gapcited by 16
  18. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  19. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  20. 2024Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Modelscited by 12
  21. 2024Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Featurescited by 11
  22. 2025RepIt: Steering Language Models with Concept-Specific Refusal Vectorscited by 10
  23. 2024Decoupled Alignment for Robust Plug-and-Play Adaptationcited by 9
  24. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 9
  25. 2024SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidancecited by 8
  26. 2025AdaBoN: Adaptive Best-of-N Alignmentcited by 8
  27. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  28. 2025Inference-time Alignment in Continuous Spacecited by 7
  29. 2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 7
  30. 2024Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasetscited by 5
  31. 2024MetaAlign: Align Large Language Models with Diverse Preferences during Inference Timecited by 5
  32. 2025Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Modelscited by 5
  33. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 4
  34. 2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gapcited by 3
  35. 2025The Anatomy of Alignment: Decomposing Preference Optimization by Steering Sparse Featurescited by 3
  36. 2026Steering to Say No: Configurable Refusal via Activation Steering in Vision Language Modelscited by 3
  37. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  38. 2024Inference time LLM alignment in single and multidomain preference spectrumcited by 2

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025In-Training Defenses against Emergent Misalignment in Language Models
  2. 2026An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS
  3. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
  4. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  5. 2025InfiAlign: A Scalable and Sample-Efficient Framework for Aligning LLMs to Enhance Reasoning Capabilities
  6. 2026SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
  7. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  8. 2025We Think, Therefore We Align LLMs to Helpful, Harmless and Honest Before They Go Wrong