# Via Negativa for AI Alignment: Why Negative Constraints Are Structurally Superior to Positive Preferences

arXiv: https://arxiv.org/abs/2603.16417 · HTML: https://mlsanity.com/paper/2603.16417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 10 of 10)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [Large Language Model Unlearning](https://mlsanity.com/paper/2310.10683/) (2023)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning](https://mlsanity.com/paper/2506.01347/) (2025)
- [How RLHF Amplifies Sycophancy](https://mlsanity.com/paper/2602.01002/) (2026)
- [Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization](https://mlsanity.com/paper/2403.03419/) (2024)
- [As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss](https://mlsanity.com/paper/2410.04834/) (2024)
- [Why the Valuable Capabilities of LLMs Are Precisely the Unexplainable Ones](https://mlsanity.com/paper/2603.15238/) (2026)

## Abstract (arXiv)

Recent empirical results have demonstrated that training large language models (LLMs) with negative-only feedback can match or exceed standard reinforcement learning from human feedback (RLHF). Negative Sample Reinforcement achieves parity with PPO on mathematical reasoning; Distributional Dispreference Optimization trains effectively using only dispreferred samples; and Constitutional AI outperforms pure RLHF on harmlessness benchmarks. Yet no unified theoretical account explains why negative signals are so effective. This paper proposes such an account: positive preferences and negative constraints are structurally asymmetric. Positive preferences ("which is better") encode continuously coupled, context-dependent human values that cannot be exhaustively specified -- leading models to learn surface correlates such as agreement with the user (sycophancy). Negative constraints ("what is wrong") encode discrete, finite, independently verifiable prohibitions that can converge to a stable boundary. This asymmetry -- rooted in Popper's falsification logic and the epistemology of negative knowledge -- explains both the sycophancy failure of preference-based RLHF and the surprising effectiveness of negative-signal methods. We argue that alignment research should shift its center of gravity from "learning what humans prefer" to "learning what humans reject," and offer testable predictions for this framework.