mlsanityData to 14 Oct 2026

Where Reliability Lives: Experimental Localisation of Behavioural Properties in an Agent System

arXiv 2609.03192 ↗First posted 2 Sep 2026cs.MAcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Reliability claims about agentic systems implicitly locate each property somewhere: in the model, or in the machinery around it. We built a system where that location is an experimental question. The subject is a persistent simulated settlement whose authoritative append-only ledger adjudicates every attempted act against world state; accepted history is the only reality. Mind, institution and world were separated before any experiment. Holding cognition fixed, we intervened on the institution's epistemic mechanisms (evidence provenance, belief availability, physical-evidence legibility); preregistered experiments refuted our central prediction twice, in opposite directions. A registered falsifier then supplied the input the geometry had denied the belief channel, a staged veridical first-hand witness, and its marginal value, non-positive throughout the witness-free phases, turned positive: 9 of 11 seeds, zero added false attribution. Holding institutional enforcement fixed, we intervened on cognition four ways: ablating the native minds' machinery, killing and resetting them mid-task, substituting a frozen frontier-LLM panel for the entire native cognition, and corrupting beliefs with trusted false testimony. Behaviour changed dramatically: one falsehood cost each trusting run about 900 futile actions and the distrusting arm none. Five pre-declared properties did not move in any tested trajectory: accepted reality stayed singular, invalid attempts were refused with typed reasons, duties outlived their processes, no work was accepted twice, and no false completion was ever accepted (2,581 substituted-panel claims, none false). Our claim is limited to this setting: measured behavioural properties were separable from substantial changes to cognition, established by intervention. One designed world, not a population of institutions; no test of an agent optimising against the institution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 54 of its 80 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

54 of 80 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026When "Must" Becomes "Maybe": Constraint Weakening in LLM Agent Workflows25 Aug
  2. 2025Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerance13 Nov
  3. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning15 Mar
  4. 2024On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents2 Aug
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun

Other references · 49

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  7. 2017Selective Classification for Deep Neural Networkscited by 852
  8. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  9. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  10. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  11. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  12. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
37 more
  1. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  2. 2025Defeating Prompt Injections by Designcited by 297
  3. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
  4. 2025Progent: Securing AI Agents with Privilege Controlcited by 141
  5. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 133
  6. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 126
  7. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 125
  8. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 124
  9. 2024Collective Constitutional AI: Aligning a Language Model with Public Inputcited by 123
  10. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  11. 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agentscited by 64
  12. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 53
  13. 2025Correlated Errors in Large Language Modelscited by 45
  14. 2026Formal Policy Enforcement for Real-World Agentic Systemscited by 33
  15. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  16. 2026Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphscited by 20
  17. 2025Governance-as-a-Service: A Multi-Agent Framework for AI System Compliance and Policy Enforcementcited by 17
  18. 2026Agent-Diff: Benchmarking LLM Agents on Enterprise API Tasks via Code Execution with State-Diff-Based Evaluationcited by 13
  19. 2026Conformity Dynamics in LLM Multi-Agent Systems: The Roles of Topology and Self-Social Weightingcited by 10
  20. 2026AgentRewind: Recoverable Execution for Long-Horizon LLM Agentscited by 7
  21. 2026Deontic Policies for Runtime Governance of Agentic AI Systemscited by 5
  22. 2026AID-Guard: Stateful Authorization for Delegated Agent Effectscited by 5
  23. 2026When Truth Is Distributed: Misinformation Derails Collective Fact Recovery in LLM-Based Multi-Agent Systemscited by 4
  24. 2026Beyond Memory: A Transactional Continuity Kernel for Long-Lived AI Agentscited by 4
  25. 2026AI Agents Alone Are Not (Yet) Sufficient for Social Simulationcited by 3
  26. 2026LogAct: Enabling Agentic Reliability via Shared Logscited by 3
  27. 2026Enforcing Benign Trajectories: A Behavioral Firewall for Structured-Workflow AI Agentscited by 3
  28. 2025Collaborative QA using Interacting LLMs. Impact of Network Structure, Node Capability and Distributed Datacited by 2
  29. 2026Hallucination as Exploit: Evidence-Carrying Multimodal Agentscited by 2
  30. 2026Toward Safe LLM Agents: A Survey of Specification, Verification, and Enforcementcited by 2
  31. 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systemscited by 2
  32. 2017Scientific Polarizationcited by 1
  33. 2018How to Beat Science and Influence People: Policy Makers and Propaganda in Epistemic Networkscited by 1
  34. 2026Preregistered Belief Revision Contractscited by 1
  35. 2026When Agents Evolve, Institutions Followcited by 1
  36. 2026Artificial Institutions: How Institutional Design Shapes LLM Simulationscited by 1
  37. 2026Multi-Agent AI Safety as an Institutional Design Problemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Stored Is Not Supported: Typed Provenance and Assertion Guardrails for Persistent AI Agents
  2. 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgents
  3. 2026Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
  4. 2026Local verification cannot detect non-transportability: a cohomological theory of context preservation in agentic reasoning
  5. 2026Active Inference: A method for Phenotyping Agency in AI systems?
  6. 2026Before Agents Decide: Epistemic Action in LLM-Based Systems
  7. 2026Epistemic Traps: Rational Misalignment Driven by Model Misspecification
  8. 2026The Epistemics of Agent Memory: Measuring, and Governing, the Consolidation Decision in Long-Horizon LLM Agents