mlsanityData to 7 Oct 2026

How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment

arXiv 2406.11474 ↗First posted 17 Jun 2024cs.CL cs.AIcited by 6

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Recent studies have demonstrated that In-Context Learning (ICL), through the use of specific demonstrations, can align Large Language Models (LLMs) with human preferences known as In-Context Alignment (ICA), indicating that models can comprehend human instructions without requiring parameter adjustments. However, the exploration of the mechanism and applicability of ICA remains limited. In this paper, we begin by dividing the context text used in ICA into three categories: format, system prompt, and example. Through ablation experiments, we investigate the effectiveness of each part in enabling ICA to function effectively. We then examine how variants in these parts impact the model's alignment performance. Our findings indicate that the example part is crucial for enhancing the model's alignment capabilities, with changes in examples significantly affecting alignment performance. We also conduct a comprehensive evaluation of ICA's zero-shot capabilities in various alignment tasks. The results indicate that compared to parameter fine-tuning methods, ICA demonstrates superior performance in knowledge-based tasks and tool-use tasks. However, it still exhibits certain limitations in areas such as multi-turn dialogues and instruction following.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 39 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

26 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning4 Dec
  2. 2023LIMA: Less Is More for Alignment18 May
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  4. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  9. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  10. 2023The False Promise of Imitating Proprietary LLMscited by 254
  11. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  12. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
10 more
  1. 2022Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Orderingcited by 158
  2. 2023Finding Support Examples for In-Context Learningcited by 104
  3. 2024A Closer Look at the Limitations of Instruction Tuningcited by 76
  4. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  5. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 71
  6. 2023Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Followingcited by 33
  7. 2023In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuningcited by 19
  8. 2023A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Taskscited by 17
  9. 2022Improving Fine-tuning of Self-supervised Models with Contrastive Initializationcited by 10
  10. 2022Parameter-Efficient Finetuning for Robust Continual Multilingual Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirementscited by 35
  2. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 31
  3. 2024The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignmentcited by 11
  4. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  5. 2025Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimizationcited by 1
  6. 2026Beyond Static Alignment: Hierarchical Policy Control for LLM Safety via Risk-Aware Chain-of-Thoughtcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
  2. 2024Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
  3. 2023Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment
  4. 2025RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars
  5. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  6. 2025PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization
  7. 2023Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
  8. 2026Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning