mlsanityData to 7 Oct 2026

pAI-Econ-claude: A Gated Human-in-the-Loop Multi-Agent Architecture for AI-Assisted Economic Theory Development

arXiv 2607.21268 ↗First posted 23 Jul 2026cs.MA cs.AI econ.GNcited by 0

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

In many social-science research tasks, such as economics, LLM-based agents must produce outputs for which no cheap, task-complete, machine-readable correctness signal exists. This creates a distinctive reliability problem for multi-agent systems: how should generation, critique, coordination, and human judgment be organized when no component can certify the final result? We address this problem through pAI-Econ-claude, a gated, human-in-the-loop multi-agent architecture for AI-assisted economic theory development. Agents coordinate through a shared workspace of inspectable intermediate records; specialized gates diagnose targeted failure modes and recommend loopbacks without certifying correctness; and human checkpoints retain authority over decisions that are costly to reverse. We evaluate the architecture on five matched economic-theory tasks against an ungated baseline. Two evaluators blinded to configuration agreed on all five pairwise rankings, preferring the gated architecture in four tasks and the baseline in one. Mean failure severity fell from 1.58 to 1.16, while overall usefulness rose from 2.60 to 3.10. The largest observed gain occurred when a reality check rejected a false market-structure premise and a proof review prompted revision of a false welfare claim. The negative case shows that scaffolding can also compress an economically important mechanism too aggressively. The results support a bounded claim: gated oversight improves the auditability of AI-assisted economic theory without substituting for formal verification, and the allocation of irreversible human judgment is a more informative design variable than pure agent autonomy. The workflow is publicly available at https://github.com/maxwell2732/pAI-Econ-claude.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 23 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

15 of 23 references.

References · 15

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  8. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  9. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  10. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  11. 2024Autonomous LLM-driven research from data to human-verifiable research paperscited by 50
  12. 2024Human-In-the-Loop Software Development Agentscited by 42
3 more
  1. 2024From Intention To Implementation: Automating Biomedical Research via LLMscited by 7
  2. 2026(Human) Attention Is (Still) All You Need: Human oversight makes AI-assisted social science reliablecited by 3
  3. 2026pAI/MSc: ML Theory Research with Humans on the Loopcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026HLER: Human-in-the-Loop Economic Research via Multi-Agent Pipelines for Empirical Discovery
  2. 2026AgentEconomist: An End-to-end Agentic System Translating Economic Intuitions into Executable Computational Experiments
  3. 2023Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models
  4. 2026AI Economist Agent: An Agentic Framework for Evidence-Based Economic and Financial Analysis with RAG, Knowledge Graphs, and Large Language Models
  5. 2025Towards Cognitive Synergy in LLM-Based Multi-Agent Systems: Integrating Theory of Mind and Critical Evaluation
  6. 2026EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents
  7. 2026Alignment as Institutional Design: From Behavioral Correction to Transaction Structure in Intelligent Systems
  8. 2026AI Coding Agents in Social Science: Methodologically Diverse, Empirically Consistent, Interpretively Vulnerable