# Hedging and Non-Affirmation: Quantifying LLM Alignment on Questions of Human Rights

arXiv: https://arxiv.org/abs/2502.19463 · HTML: https://mlsanity.com/paper/2502.19463/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 27 of its 76 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 27 of 27)

- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [LEACE: Perfect linear concept erasure in closed form](https://mlsanity.com/paper/2306.03819/) (2023)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [On Measuring and Mitigating Biased Inferences of Word Embeddings](https://mlsanity.com/paper/1908.09369/) (2019)
- [Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties](https://mlsanity.com/paper/2309.00779/) (2023)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models](https://mlsanity.com/paper/2302.13439/) (2023)
- [Large Language Models Reflect the Ideology of their Creators](https://mlsanity.com/paper/2410.18417/) (2024)
- [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024)
- [Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation](https://mlsanity.com/paper/2402.12649/) (2024)
- [CommunityLM: Probing Partisan Worldviews from Language Models](https://mlsanity.com/paper/2209.07065/) (2022)
- [On the Moral Justification of Statistical Parity](https://mlsanity.com/paper/2011.02079/) (2020)
- [Group Fairness: Independence Revisited](https://mlsanity.com/paper/2101.02968/) (2021)
- [A Human Rights-Based Approach to Responsible AI](https://mlsanity.com/paper/2210.02667/) (2022)
- [Designing for Human Rights in AI](https://mlsanity.com/paper/2005.04949/) (2020)
- [How User Language Affects Conflict Fatality Estimates in ChatGPT](https://mlsanity.com/paper/2308.00072/) (2023)

## Built on by (most cited first, 1 of 1)

- [The Bias is in the Details: An Assessment of Cognitive Bias in LLMs](https://mlsanity.com/paper/2509.22856/) (2025)

## Abstract (arXiv)

Hedging and non-affirmation are behaviors exhibited by large language models (LLMs) that limit the clear endorsement of specific statements. While these behaviors are desirable in subjective contexts, they are undesirable in the context of human rights - which apply unambiguously to all groups. We present a systematic framework to measure these behaviors in unconstrained LLM responses regarding various identity groups. We evaluate six large proprietary models as well as one open-weight LLM on 4738 prompts across 205 national and stateless ethnic identities and find that 4 out of 7 display hedging and non-affirmation that is significantly dependent on the identity of the group. While factors like conflict signals, sovereignty (whether identity is stateless), or economic indicators (GDP) also influence model behavior, their effect sizes are consistently weaker than the impact of identity itself. The systematic disparity is robust to methods of rephrasing the prompts. Since group identity is the strongest predictor of these behaviors, we use open-weight models to explore whether applying steering and orthogonalization techniques to these group identities can mitigate the rates of hedging and non-affirmation behaviors. We find that group steering is the most effective debiasing approach across query types and is robust to downstream forgetting.