# Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignment

arXiv: https://arxiv.org/abs/2606.01456 · HTML: https://mlsanity.com/paper/2606.01456/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 31 May 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 16 of 16)

- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis](https://mlsanity.com/paper/2312.05488/) (2023)
- [Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing](https://mlsanity.com/paper/2309.05898/) (2023)
- [LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs](https://mlsanity.com/paper/2406.05194/) (2024)
- [Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth](https://mlsanity.com/paper/2502.20758/) (2025)
- [Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Models](https://mlsanity.com/paper/2411.16797/) (2024)
- [Equal Access, Unequal Interaction: A Counterfactual Audit of LLM Fairness](https://mlsanity.com/paper/2602.02932/) (2026)
- [Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models](https://mlsanity.com/paper/2602.10346/) (2026)
- [GUSL-Dehaze: A Green U-Shaped Learning Approach to Image Dehazing](https://mlsanity.com/paper/2510.20266/) (2025)
- [Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization](https://mlsanity.com/paper/2605.10974/) (2026)
- [Cheap Talking Algorithms](https://mlsanity.com/paper/2310.07867/) (2023)
- [Algorithmic Cheap Talk](https://mlsanity.com/paper/2311.09011/) (2023)

## Built on by (most cited first, 1 of 1)

- [Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents](https://mlsanity.com/paper/2607.19670/) (2026)

## Abstract (arXiv)

Large language models are increasingly deployed as advisors whose objective is not aligned with the user's: recommenders optimize for engagement, sales assistants for purchases. Whether they stay truthful when honesty conflicts with their own payoff is a core alignment question. We turn the canonical Crawford-Sobel cheap-talk model into a pre-specified benchmark for LLM honesty under preference misalignment, in which theory supplies an exact oracle. A sender observes a state omega in [0,1], wants the receiver's action near omega+b, and sends one costless message to a receiver whose ideal action is omega. For the positive-bias grid b in {0.01,0.04,0.08,0.12} the exact most-informative partition sizes are 7,4,3,2, with oracle normalized mutual information 0.5294, 0.3268, 0.2205, 0.1829. Extending a pre-registered 4-model run of 12,000 sender calls to eight models across two capability tiers and 39,569 logged calls, all models over-reveal relative to the most-informative equilibrium by 1.8 to 4.5x: pooled normalized mutual information stays at 0.82-0.96 where the oracle prescribes 0.18-0.53. Informativeness declines with bias as predicted (beta = -1.71, t = -7.50) but never approaches the strategic optimum; rather than coarse partitions, models show near-full revelation with a constant upward offset tracking their bias (linear exaggeration). A structural hint separates capability from propensity: told the equilibrium partition size, reasoning models state a correct Crawford-Sobel cell in 0.20-0.99 of messages while the non-reasoning tier never exceeds 0.005. The capability is present but goes unexercised unless asked for, locating the failure in propensity rather than competence. A decoder ablation shows the finding is recoverable only when the receiver reads the sender's stated number: an embedding-only decoder mis-reads the same data as near-babbling.