# The AI Alignment Paradox

arXiv: https://arxiv.org/abs/2405.20806 · HTML: https://mlsanity.com/paper/2405.20806/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 8 of 8)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Fundamental Limitations of Alignment in Large Language Models](https://mlsanity.com/paper/2304.11082/) (2023)
- [JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs](https://mlsanity.com/paper/2402.05668/) (2024)
- [Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space](https://mlsanity.com/paper/2402.09063/) (2024)
- [The Radicalization Risks of GPT-3 and Advanced Neural Language Models](https://mlsanity.com/paper/2009.06807/) (2020)

## Built on by (most cited first, 1 of 1)

- [From Model Training to Model Raising](https://mlsanity.com/paper/2511.09287/) (2025)

## Abstract (arXiv)

The field of AI alignment aims to steer AI systems toward human goals, preferences, and ethical principles. Its contributions have been instrumental for improving the output quality, safety, and trustworthiness of today's AI models. This perspective article draws attention to a fundamental challenge we see in all AI alignment endeavors, which we term the "AI alignment paradox": The better we align AI models with our values, the easier we may make it for adversaries to misalign the models. We illustrate the paradox by sketching three concrete example incarnations for the case of language models, each corresponding to a distinct way in which adversaries might exploit the paradox. With AI's increasing real-world impact, it is imperative that a broad community of researchers be aware of the AI alignment paradox and work to find ways to mitigate it, in order to ensure the beneficial use of AI for the good of humanity.