mlsanityData to 7 Oct 2026

Hedging and Non-Affirmation: Quantifying LLM Alignment on Questions of Human Rights

arXiv 2502.19463 ↗First posted 26 Feb 2025cs.CY cs.AI cs.SIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Hedging and non-affirmation are behaviors exhibited by large language models (LLMs) that limit the clear endorsement of specific statements. While these behaviors are desirable in subjective contexts, they are undesirable in the context of human rights - which apply unambiguously to all groups. We present a systematic framework to measure these behaviors in unconstrained LLM responses regarding various identity groups. We evaluate six large proprietary models as well as one open-weight LLM on 4738 prompts across 205 national and stateless ethnic identities and find that 4 out of 7 display hedging and non-affirmation that is significantly dependent on the identity of the group. While factors like conflict signals, sovereignty (whether identity is stateless), or economic indicators (GDP) also influence model behavior, their effect sizes are consistently weaker than the impact of identity itself. The systematic disparity is robust to methods of rephrasing the prompts. Since group identity is the strongest predictor of these behaviors, we use open-weight models to explore whether applying steering and orthogonalization techniques to these group identities can mitigate the rates of hedging and non-affirmation behaviors. We find that group steering is the most effective debiasing approach across query types and is robust to downstream forgetting.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 76 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

27 of 76 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Collective Constitutional AI: Aligning a Language Model with Public Input12 Jun

Other references · 26

  1. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  2. 2023Steering Language Models With Activation Engineeringcited by 824
  3. 2023Whose Opinions Do Language Models Reflect?cited by 759
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  5. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  7. 2020Artificial Intelligence, Values and Alignmentcited by 443
  8. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  9. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  10. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  11. 2021Alignment of Language Agentscited by 167
  12. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
14 more
  1. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  2. 2019On Measuring and Mitigating Biased Inferences of Word Embeddingscited by 112
  3. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  4. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  5. 2024Large Language Models are Geographically Biasedcited by 91
  6. 2024Large Language Models Reflect the Ideology of their Creatorscited by 56
  7. 2024Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them incited by 45
  8. 2024Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluationcited by 41
  9. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
  10. 2020On the Moral Justification of Statistical Paritycited by 30
  11. 2021Group Fairness: Independence Revisitedcited by 20
  12. 2022A Human Rights-Based Approach to Responsible AIcited by 20
  13. 2020Designing for Human Rights in AIcited by 13
  14. 2023How User Language Affects Conflict Fatality Estimates in ChatGPTcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025The Bias is in the Details: An Assessment of Cognitive Bias in LLMscited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2025Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes
  2. 2025Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models
  3. 2025Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?
  4. 2025Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases
  5. 2024With a Grain of SALT: Are LLMs Fair Across Social Dimensions?
  6. 2025Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs
  7. 2023Exploring the psychology of LLMs' Moral and Legal Reasoning
  8. 2026Collective Opinion Dynamics in Structured LLM Populations