mlsanityData to 14 Oct 2026

A Survey on Training-free Alignment of Large Language Models

arXiv 2508.09016 ↗First posted 12 Aug 2025cs.CL cs.LGcited by 2

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

The alignment of large language models (LLMs) aims to ensure their outputs adhere to human values, ethical standards, and legal norms. Traditional alignment methods often rely on resource-intensive fine-tuning (FT), which may suffer from knowledge degradation and face challenges in scenarios where the model accessibility or computational resources are constrained. In contrast, training-free (TF) alignment techniques--leveraging in-context learning, decoding-time adjustments, and post-generation corrections--offer a promising alternative by enabling alignment without heavily retraining LLMs, making them adaptable to both open-source and closed-source environments. This paper presents the first systematic review of TF alignment methods, categorizing them by stages of pre-decoding, in-decoding, and post-decoding. For each stage, we provide a detailed examination from the viewpoint of LLMs and multimodal LLMs (MLLMs), highlighting their mechanisms and limitations. Furthermore, we identify key challenges and future directions, paving the way for more inclusive and effective TF alignment techniques. By synthesizing and organizing the rapidly growing body of research, this survey offers a guidance for practitioners and advances the development of safer and more reliable LLMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 79 of its 106 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

79 of 106 references.

Closest ancestors · 22

Same direction, newest first

  1. 2025P-Aligner: Enabling Pre-Alignment of Language Models via Principled Instruction Synthesis6 Aug
  2. 2025Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding9 Jun
  3. 2025On-the-fly Preference Alignment via Principle-Guided Decoding20 Feb
  4. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling18 Oct
  5. 2024GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment10 Oct
  6. 2024Inference-Time Language Model Alignment via Integrated Value Guidance26 Sep
  7. 2024Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts9 Aug
  8. 2024Decoding-Time Language Model Alignment with Multiple Objectives27 Jun
  9. 2024Cascade Reward Sampling for Efficient Decoding-Time Alignment24 Jun
  10. 2024Inference Time Alignment with Reward-Guided Tree Search21 Jun
  11. 2024Decoding-time Realignment of Language Models5 Feb
  12. 2024Aligner: Efficient Alignment by Learning to Correct4 Feb
10 more
  1. 2024ARGS: Alignment as Reward-Guided Search23 Jan
  2. 2024Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback21 Jan
  3. 2023Align on the Fly: Adapting Chatbot Behavior to Established Norms26 Dec
  4. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning4 Dec
  5. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Training7 Nov
  6. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages7 Oct
  7. 2023Large Language Model Alignment: A Survey26 Sep
  8. 2023RAIN: Your Language Models Can Align Themselves without Finetuning13 Sep
  9. 2023LIMA: Less Is More for Alignment18 May
  10. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 57

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  12. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
38 more
  1. 2020Artificial Intelligence, Values and Alignmentcited by 448
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  3. 2022Prompting GPT-3 To Be Reliablecited by 260
  4. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  5. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  6. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  7. 2024Investigating Cultural Alignment of Large Language Modelscited by 143
  8. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  9. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 124
  10. 2023Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languagescited by 97
  11. 2024Reinforcement Learning for LLM Post-Training: A Surveycited by 92
  12. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  13. 2023Moral Foundations of Large Language Modelscited by 85
  14. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83
  15. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignmentcited by 71
  16. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 60
  17. 2024PAD: Personalized Alignment of LLMs at Decoding-Timecited by 60
  18. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 60
  19. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 50
  20. 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
  21. 2023Self-Detoxifying Language Models via Toxification Reversalcited by 46
  22. 2024Autoregressive Models in Vision: A Surveycited by 39
  23. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  24. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 36
  25. 2023Can large language models democratize access to dual-use biotechnology?cited by 35
  26. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibrationcited by 31
  27. 2024Domain Generalization through Meta-Learning: A Surveycited by 28
  28. 2024Large Language Models Show Human-like Social Desirability Biases in Survey Responsescited by 28
  29. 2023Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learningcited by 26
  30. 2024Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neuronscited by 25
  31. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  32. 2024AutoCAP: Towards Automatic Cross-lingual Alignment Planning for Zero-shot Chain-of-Thoughtcited by 19
  33. 2024HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Modelscited by 19
  34. 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Promptingcited by 19
  35. 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wildcited by 18
  36. 2023Improving Large Language Models in Event Relation Logical Predictioncited by 17
  37. 2024Empowering Large Language Models for Textual Data Augmentationcited by 17
  38. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gapcited by 17

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Emergent Causal-Geometric Dynamics Across Depth in Large Language Modelscited by 0
  2. 2026Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  2. 2024Efficient Alignment of Large Language Models via Data Sampling
  3. 2025RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars
  4. 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs
  5. 2023Aligning Large Language Models with Human: A Survey
  6. 2024DeAL: Decoding-time Alignment for Large Language Models
  7. 2024Aligners: Decoupling LLMs and Alignment
  8. 2025Towards Efficient and Effective Alignment of Large Language Models