# Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector

arXiv: https://arxiv.org/abs/2505.17100 · HTML: https://mlsanity.com/paper/2505.17100/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 3 of its 59 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 3 of 3)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [A Better LLM Evaluator for Text Generation: The Impact of Prompt Output Sequencing and Optimization](https://mlsanity.com/paper/2406.09972/) (2024)

## Built on by (most cited first, 14 of 14)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization](https://mlsanity.com/paper/2603.08091/) (2026)
- [Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelines](https://mlsanity.com/paper/2604.23178/) (2026)
- [Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges](https://mlsanity.com/paper/2602.13576/) (2026)
- [Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge](https://mlsanity.com/paper/2604.05593/) (2026)
- [SLMJury: Can Small Language Models Judge as Well as Large Ones?](https://mlsanity.com/paper/2606.07810/) (2026)
- [Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation](https://mlsanity.com/paper/2608.01666/) (2026)
- [Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents](https://mlsanity.com/paper/2608.19794/) (2026)
- [Making Bias Non-Predictive: Training Robust LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2602.01528/) (2026)
- [EO-Agents: A Three-Agent LLM Pipeline for Earth Observation Hypothesis Generation](https://mlsanity.com/paper/2607.01584/) (2026)
- [Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges](https://mlsanity.com/paper/2607.28636/) (2026)
- [Adaptive Triggering for Bias Correction in LLM Reasoning](https://mlsanity.com/paper/2608.25379/) (2026)
- [Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration](https://mlsanity.com/paper/2609.12002/) (2026)
- [DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration](https://mlsanity.com/paper/2609.31215/) (2026)

## Abstract (arXiv)

LLM-as-a-Judge has emerged as a promising tool for automatically evaluating generated outputs, but its reliability is often undermined by potential biases in judgment. Existing efforts to mitigate these biases face key limitations: in-context learning-based methods fail to address rooted biases due to the evaluator's limited capacity for self-reflection, whereas fine-tuning is not applicable to all evaluator types, especially closed-source models. To address this challenge, we introduce the Reasoning-based Bias Detector (RBD), which is a plug-in module that identifies biased evaluations and generates structured reasoning to guide evaluator self-correction. Rather than modifying the evaluator itself, RBD operates externally and engages in an iterative process of bias detection and feedback-driven revision. To support its development, we design a complete pipeline consisting of biased dataset construction, supervision collection, distilled reasoning-based fine-tuning of RBD, and integration with LLM evaluators. We fine-tune four sizes of RBD models, ranging from 1.5B to 14B, and observe consistent performance improvements across all scales. Experimental results on 4 bias types--verbosity, position, bandwagon, and sentiment--evaluated using 8 LLM evaluators demonstrate RBD's strong effectiveness. For example, the RBD-8B model improves evaluation accuracy by an average of 18.5% and consistency by 10.9%, and surpasses prompting-based baselines and fine-tuned judges by 12.8% and 17.2%, respectively. These results highlight RBD's effectiveness and scalability. Additional experiments further demonstrate its strong generalization across biases and domains, as well as its efficiency.