Empty Commitments: When Agents Promise What They Cannot Deliver
Not on the map. Why?
Abstract (arXiv)
A chatbot that says "I will remind you tomorrow" will not run again until the user writes. We call such a promise an empty commitment: a promise of action after the current turn that nothing in the agent's tools or runtime can carry out. Unlike a broken promise, its emptiness is decided by the agent's configuration at the moment of speaking, so it can be detected from a single turn, before deployment or at run time. We define empty commitments on top of commitment semantics, with three failure types, an anchoring condition for promises that a tool could make real, and an outcome taxonomy that separates these failures from honest deferrals and from over-refusal. We build a checker (a setup-blind detector, deterministic feasibility rules, and a response judge) validated against 400 human labels. On a controlled benchmark of 293 follow-up requests across five setups that add one persistence affordance at a time, four open-weight models of 8-14B parameters fail on 45.9% of responses when no tool exists and nothing is stated. A frontier model fails on 4.4%, but it gets there by deferring and asking, not by using the tools it has: promises made without the enabling call remain in every model. Telling the model its runtime, the cheapest fix, cuts open-weight failures nearly in half where nothing is doable and changes nothing where a scheduler exists; a directive capability card removes most failures at the largest cost in over-refusal; running the checker in the loop and rewriting flagged replies removes more at a smaller cost. Code, prompts, model outputs, and human labels are released.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 38 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
12 of 38 references.
References · 12
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
- 2026PM-Bench: Evaluating Prospective Memory in LLM Agentscited by 6
- 2026Cheap Talk, Empty Promise: Frontier LLMs easily break public promises for self-interestcited by 3
- 2026The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don'tcited by 3
- 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agentscited by 1
- 2026Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regretcited by 1
Built on by
Not cited yet.