mlsanityData to 14 Oct 2026

Mitigating Self-Preference by Authorship Obfuscation

arXiv 2512.05379 ↗First posted 5 Dec 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) judges are widely used to evaluate the quality of LM outputs. Despite many advantages, LM judges display concerning biases that can impair their integrity in evaluations. One such bias is self-preference: LM judges preferring their own answers over those produced by other LMs or humans. The bias is hard to eliminate as frontier LM judges can distinguish their own outputs from those of others, even when the evaluation candidates are not labeled with their sources. In this paper, we investigate strategies to mitigate self-preference by reducing the LM judges' ability to recognize their own outputs. We apply black-box perturbations to evaluation candidates in pairwise comparison to obfuscate the authorship and reduce self-recognition. We find that perturbations as simple as synonym replacement for a few words predictably reduce self-preference. However, we also uncover fundamental challenges to eliminating the bias: when we extrapolate our perturbations to a more complete neutralization of stylistic differences between the evaluation candidates, self-preference recovers. Our findings suggest that self-recognition and self-preference can happen on many semantic levels, and complete mitigation remains challenging despite promising initial results.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 5 Dec 2025.

Builds on

13 of 18 references.

References · 13

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  4. 2023Teaching Large Language Models to Self-Debugcited by 988
  5. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  6. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  7. 2022Self-critiquing models for assisting human evaluatorscited by 380
  8. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  9. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
  10. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  11. 2025Great Models Think Alike and this Undermines AI Oversightcited by 58
  12. 2025Do LLM Evaluators Prefer Themselves for a Reason?cited by 38
1 more
  1. 2024TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methodscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learningcited by 32
  2. 2026Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluationscited by 5
  3. 2026When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVRcited by 4
  4. 2026Self- and Other-Labels Induce Bidirectional Bias in LLM Judgescited by 1
  5. 2026Sharding Prevents LLM Oversight Failures and Adversarial Exploitationcited by 0
  6. 2026Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibrationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Evaluators Recognize and Favor Their Own Generations
  2. 2025Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
  3. 2026Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
  4. 2024Self-Preference Bias in LLM-as-a-Judge
  5. 2026Self-Preference Is Weak or Absent in Verifiable Instruction-Following Revision: A Four-Model Test Under Genuine Authorship
  6. 2025Beyond the Surface: Measuring Self-Preference in LLM Judgments
  7. 2025Semantic Voting: A Self-Evaluation-Free Approach for Efficient LLM Self-Improvement on Unverifiable Open-ended Tasks
  8. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement