# Exploring the Effects of Alignment on Numerical Bias in Large Language Models

arXiv: https://arxiv.org/abs/2601.16444 · HTML: https://mlsanity.com/paper/2601.16444/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Builds on (most cited first, 12 of 12)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Large Language Models are Inconsistent and Biased Evaluators](https://mlsanity.com/paper/2405.01724/) (2024)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [Creativity Has Left the Chat: The Price of Debiasing Language Models](https://mlsanity.com/paper/2406.05587/) (2024)
- [CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data](https://mlsanity.com/paper/2003.07008/) (2020)
- [Generative Calibration for In-context Learning](https://mlsanity.com/paper/2310.10266/) (2023)

## Built on by (most cited first, 3 of 3)

- [No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions](https://mlsanity.com/paper/2606.13044/) (2026)
- [Mitigating Scoring Bias in LLM-as-a-Judge via Random Number Generation](https://mlsanity.com/paper/2608.05726/) (2026)
- [Agent-based Modeling: Equilibrium, Echo Chambers, and Efficiency in Hybrid Coevolutionary Opinion Games](https://mlsanity.com/paper/2609.27639/) (2026)

## Abstract (arXiv)

"LLM-as-a-judge," which utilizes large language models (LLMs) as evaluators, has proven effective in many evaluation tasks. However, evaluator LLMs exhibit numerical bias, a phenomenon where certain evaluation scores are generated disproportionately often, leading reduced evaluation performance. This study investigates the cause of this bias. Given that most evaluator LLMs are aligned through instruction tuning and preference tuning, and that prior research suggests alignment reduces output diversity, we hypothesize that numerical bias arises from alignment. To test this, we compare outputs from pre- and post-alignment LLMs, and observe that alignment indeed increases numerical bias. We also explore mitigation strategies for post-alignment LLMs, including temperature scaling, distribution calibration, and score range adjustment. Among these, score range adjustment is most effective in reducing bias and improving performance, though still heuristic. Our findings highlight the need for further work on optimal score range selection and more robust mitigation strategies.