When Is Delegated Play Truthful? Within-Range Regret and the Trilemma of Aligned Delegation
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Advertisers delegate bidding to autobidders; users delegate tasks to language-model agents. A person describes what they want to an automated proxy that acts in a mechanism on their behalf. This is the revelation principle in production, and it forces a question classical theory assumes away: when is it optimal to describe yourself honestly to your own proxy? We show the answer turns on one quantity, the proxy's within-range regret. The most a principal can gain by misreporting equals the regret of the proxy's honest-report action against those the principal could have steered it to take. Honest self-description is optimal exactly when the proxy already plays the best action it can reach, that is, when it is loyal (Theorem 1). The identity unifies auction-specific autobidding results and pins down when the faithful-communication assumption behind language-model elicitation proxies (Huang et al.) holds. The identity constrains guardrails placed on proxies, from bid caps to a model's alignment layer. No guardrail can be at once binding (it displaces the truthful action from the proxy's best reachable outcome), truthful (honest reporting stays optimal), and capability-preserving (that outcome stays reachable through some report); any two preclude the third (Theorem 2). A safety constraint that alters what a model does while leaving its best output reachable makes honest description of intent suboptimal, so a sharper report can gain. This is the incentive behind prompt-engineering and jailbreaking. Because within-range regret is #P-hard to compute exactly, we estimate it from samples and maintain it as a model is updated, at a cost set by how far the model drifts, not how often it changes. Running it on production language models from five providers under an alignment-style cap, we find honest reporting leaves surplus unclaimed on every model, recovered by inflating the report.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. First posted on arXiv 15 Jul 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games
- 2026The Confidence Game: Strategic Miscalibration in Human-AI Delegation
- 2026Paying for Honesty Without Knowing the Truth: Reputation-Penalty Design for LLM Marketplace Agents
- 2026Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
- 2026Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives
- 2026Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
- 2026Capability Advertisement as a Market for Lemons: A Trust Layer for Heterogeneous Agent Networks
- 2026Nudgeability: Reasoning Models Follow Confidence Signals Without Tracking Their Own Competence