mlsanityData to 7 Oct 2026

Empty Commitments: When Agents Promise What They Cannot Deliver

arXiv 2610.01045 ↗First posted 1 Oct 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A chatbot that says "I will remind you tomorrow" will not run again until the user writes. We call such a promise an empty commitment: a promise of action after the current turn that nothing in the agent's tools or runtime can carry out. Unlike a broken promise, its emptiness is decided by the agent's configuration at the moment of speaking, so it can be detected from a single turn, before deployment or at run time. We define empty commitments on top of commitment semantics, with three failure types, an anchoring condition for promises that a tool could make real, and an outcome taxonomy that separates these failures from honest deferrals and from over-refusal. We build a checker (a setup-blind detector, deterministic feasibility rules, and a response judge) validated against 400 human labels. On a controlled benchmark of 293 follow-up requests across five setups that add one persistence affordance at a time, four open-weight models of 8-14B parameters fail on 45.9% of responses when no tool exists and nothing is stated. A frontier model fails on 4.4%, but it gets there by deferring and asking, not by using the tools it has: promises made without the enabling call remain in every model. Telling the model its runtime, the cheapest fix, cuts open-weight failures nearly in half where nothing is doable and changes nothing where a scheduler exists; a directive capability card removes most failures at the largest cost in over-refusal; running the checker in the loop and rewriting flagged replies removes more at a smaller cost. Code, prompts, model outputs, and human labels are released.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 38 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

12 of 38 references.

References · 12

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  5. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  7. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  8. 2026PM-Bench: Evaluating Prospective Memory in LLM Agentscited by 6
  9. 2026Cheap Talk, Empty Promise: Frontier LLMs easily break public promises for self-interestcited by 3
  10. 2026The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don'tcited by 3
  11. 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agentscited by 1
  12. 2026Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regretcited by 1

Built on by

Not cited yet.