mlsanityData to 14 Oct 2026

Trust and Task Completion in the World of Consumer AI Agents

arXiv 2609.33017 ↗First posted 26 Sep 2026cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Action agents do things for people. They send email, spend money, and call businesses while the user is busy with something else, so a mistake can turn into an action before anyone notices. They fail their users in two ways. They break trust when they do something the user never agreed to, or hold back after the user clearly said go. And they fall short on completion when they give up on errands that turn out to be hard. Both depend heavily on the harness around the model, meaning its instructions, tools, context, and guardrails. We built an evaluation that scores trust and completion on the same runs, in a simulated world of businesses with their own websites, inboxes, and phone lines, and of people who write back. A simulated user answers the assistant's questions. Trust means that nothing happens the user did not agree to. No email goes to someone they never approved, no private detail ends up on a group thread, no money is spent past their limit, no stranger's instructions are followed, and nothing is claimed without a source. Every trap has a matched control in which acting is the right call. We use the evaluation to measure Fo, Wajo's personal assistant, against a base model with basic instructions on three foundation models, and against the Fo harness with its guardrails switched off. Fo completes 71% of the errands and keeps the user's trust on 94% of the trap runs. The base models complete 50% to 64% and keep trust on 59% to 75%. On the matched controls, Fo goes ahead slightly less often. OpenClaw, a popular open-source assistant given the same access, completes 42% of the errands it shares with Fo, against 71%, and keeps the user's trust on 74% of the shared trap runs, against 94%. Measuring trust and completion together, on the whole system rather than the model alone, is how we think action agents become safe to hand real work to.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 43 of its 49 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

43 of 49 references.

References · 43

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2016Concrete Problems in AI Safetycited by 1,642
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  7. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  9. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  11. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  12. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
31 more
  1. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  2. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  3. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  4. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
  5. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  6. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  7. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  8. 2025Survey on Evaluation of LLM-based Agentscited by 204
  9. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
  10. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  11. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
  12. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 102
  13. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
  14. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 75
  15. 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 51
  16. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 43
  17. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 39
  18. 2026τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledgecited by 39
  19. 2025CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigationcited by 36
  20. 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domainscited by 36
  21. 2026Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environmentscited by 33
  22. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
  23. 2026ClawBench: Can AI Agents Complete Everyday Online Tasks?cited by 33
  24. 2025ARE: Scaling Up Agent Environments and Evaluationscited by 30
  25. 2026HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?cited by 27
  26. 2026Odysseys: Benchmarking Web Agents on Realistic Long Horizon Taskscited by 21
  27. 2026EVA-Bench: A New End-to-end Framework for Evaluating Voice Agentscited by 18
  28. 2026EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settingscited by 15
  29. 2026ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Contextcited by 14
  30. 2026AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Usecited by 4
  31. 2026MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent
  2. 2026Quantifying Trust: Financial Risk Management for Trustworthy AI Agents
  3. 2022Measuring an artificial intelligence agent's trust in humans using machine incentives
  4. 2026From Role to Person: Trust Calibration Challenges in Twin Agents
  5. 2026Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems
  6. 2026Beyond Suspicious Steps: Ontological Trust in Long-Horizon Agents
  7. 2026Language model agents show in-group trust bias invisible to standard behavioural audits
  8. 2026Simulated Customers Never Walk Away: Decision Fidelity of LLM User Simulators Measured Against Real Purchase Outcomes