Where Knowledge and Authority Sit Changes What an Agent Benchmark Can Resolve
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Most agent benchmarks put facts, tools and permissions behind one interface. Real organizations spread them across people. Incognita asks what happens when the task and success criterion stay fixed but access does not. We transform eighteen customer-service tasks into three settings: direct access, one known intermediary, and six role-isolated participants whose capabilities must be discovered. Across 864 trials with four models, social access reduced success for every model; the pre-specified intervals excluded zero for two. The latest tested model, gpt-5.6-sol, achieved the highest social-access success at 0.65, a 0.11 decrease from centralized indirect access with an interval that included zero. Exploratory comparisons separated five of six model pairs under social access, while neither centralized setting separated any at this sample size. In post-hoc task-blocked tests, three pairwise interaction p-values remained significant after multiplicity adjustment. A reference-relative reader associates the wider gaps with failures to obtain needed information. Because the data cannot distinguish ineffective requests by the evaluated agent from inaccurate replies by simulated participants, the reader's labels describe where trajectories stopped, not why models differed.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 25 of its 36 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
25 of 36 references.
Closest ancestors · 9
Same direction, newest first
- 2026CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments4 Jun
- 2026Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows9 May
- 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems1 Mar
- 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs15 May
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
- 2024Autonomous Agents for Collaborative Task under Information Asymmetry21 Jun
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
- 2024Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents14 Feb
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct
Other references · 16
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
- 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
- 2024AI Agents That Mattercited by 219
- 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 148
- 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 126
- 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
4 more
- 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 63
- 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 54
- 2026TeamBench: Evaluating Agent Coordination under Enforced Role Separationcited by 10
- 2026CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagementcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Context Access Divide: Interaction-Level Architecture as a Complementary Dimension of Agentic Inequality
- 2025Language Model Agents Under Attack: A Cross Model-Benchmark of Profit-Seeking Behaviors in Customer Service
- 2026CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments
- 2026HoosierHelp: Benchmarking LLM Agents for Social Service Navigation
- 2026Evaluation of Agents under Simulated AI Marketplace Dynamics
- 2026Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents
- 2024WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting
- 2026The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean