mlsanityData to 7 Oct 2026

Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?

arXiv 2606.31371 ↗First posted 30 Jun 2026cs.LG cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

When large language model (LLM) agents adapt their behavior through evaluator feedback, systematic evaluator biases propagate into the agent's learned strategy distribution - a phenomenon termed evaluator preference coupling. Prior work has documented this coupling and established a diagnostic framework (EPC) to measure it, but has not investigated whether calibration techniques can mitigate the effect. We present the first study of evaluator calibration as mitigation: applying probability calibration to the evaluator's pairwise judgments to reduce spurious preference propagation. In a controlled within-subjects experiment (N=5) comparing standard binary TTRL (win/loss) with confidence-calibrated TTRL (probability-weighted updates) using DeepSeek-V4-Pro as executor and GLM5.2 as evaluator, we find that calibration reduces the coupling coefficient gamma by 20-49% and Jensen-Shannon divergence by 45-67%. A symmetric-LR control confirms the effect is not due to reduced update asymmetry. We release the calibrated TTRL protocol and recommend it as a lightweight mitigation for LLM-as-judge deployment pipelines.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 9 of its 16 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

9 of 16 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents29 Jun

Other references · 8

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 198
  4. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
  5. 2024InfAlign: Inference-aware language model alignmentcited by 34
  6. 2026Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelinescited by 8
  7. 2025Judging with Confidence: Calibrating Autoraters to Preference Distributionscited by 3
  8. 2026Uncertainty-Aware Reward Discounting for Mitigating Reward Hackingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EPC: A Standardized Protocol for Measuring Evaluator Preference Dynamics in LLM Agent Systems
  2. 2026PADMÉ: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators
  3. 2026Contagion Networks: Evaluator Preference Propagation in Multi-Agent LLM Systems
  4. 2024Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation
  5. 2025Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
  6. 2026Calibration Collapse Under Sycophancy Fine-Tuning: How Reward Hacking Breaks Uncertainty Quantification in LLMs
  7. 2026How Sampling Shapes LLM Alignment: From One-Shot Optima to Iterative Dynamics
  8. 2025Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation