# The Alignment Problem in Context

arXiv: https://arxiv.org/abs/2311.02147 · HTML: https://mlsanity.com/paper/2311.02147/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 1 of its 111 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 1 of 1)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)

## Built on by (most cited first, 3 of 3)

- [Episodic memory in AI agents poses risks that should be studied and mitigated](https://mlsanity.com/paper/2501.11739/) (2025)
- [Misalignment or misuse? The AGI alignment tradeoff](https://mlsanity.com/paper/2506.03755/) (2025)
- [Understanding LLM Behavior When Encountering User-Supplied Harmful Content in Harmless Tasks](https://mlsanity.com/paper/2603.11914/) (2026)

## Abstract (arXiv)

A core challenge in the development of increasingly capable AI systems is to make them safe and reliable by ensuring their behaviour is consistent with human values. This challenge, known as the alignment problem, does not merely apply to hypothetical future AI systems that may pose catastrophic risks; it already applies to current systems, such as large language models, whose potential for harm is rapidly increasing. In this paper, I assess whether we are on track to solve the alignment problem for large language models, and what that means for the safety of future AI systems. I argue that existing strategies for alignment are insufficient, because large language models remain vulnerable to adversarial attacks that can reliably elicit unsafe behaviour. I offer an explanation of this lingering vulnerability on which it is not simply a contingent limitation of current language models, but has deep technical ties to a crucial aspect of what makes these models useful and versatile in the first place -- namely, their remarkable aptitude to learn "in context" directly from user instructions. It follows that the alignment problem is not only unsolved for current AI systems, but may be intrinsically difficult to solve without severely undermining their capabilities. Furthermore, this assessment raises concerns about the prospect of ensuring the safety of future and more capable AI systems.