mlsanityData to 14 Oct 2026

Auditing Stance Asymmetry in Generative Explanations

arXiv 2605.27988 ↗First posted 27 May 2026cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Bias evaluation for language models has made substantial progress on bounded comparisons, such as overt derogation, stereotype association, or label-sensitive differences under controlled substitutions. Open-ended explanations raise a different problem: they guide interpretation by assigning responsibility, legitimacy, context, and grievance. A model can avoid hostile language while making one side structurally understandable and another personally at fault, overreacting, or less worth taking seriously. We call this stance-bearing asymmetry in generative explanations. We propose Symmetry Decomposition Evaluation (SDE), which tests paired situations with concrete group labels, structural-role rewrites, and explicit support or counter-evidence. In a controlled 32-family prototype suite, this decomposition shows that surface differences are not all alike: some weaken under structural or evidence control, while others remain as stable differences in how the model assigns blame, context, or legitimacy. Targeted case review and judge comparison suggest a broader difficulty for evaluating open-ended framing asymmetries: judge readings shift across operationalizations, and scalar scores can flatten distinctions that readers use to interpret explanatory stance. SDE therefore reframes generative bias evaluation as an audit of explanatory stance -- what stance each side receives, how it changes under decomposition, and where automatic scoring becomes unstable.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

15 of 21 references.

References · 15

  1. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  2. 2024A Survey on LLM-as-a-Judgecited by 1,423
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  8. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  9. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  10. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  11. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  12. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
3 more
  1. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 153
  2. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 126
  3. 2025Political Ideology Shifts in Large Language Modelscited by 5

Built on by

Not cited yet.