# MarketBench: Evaluating AI Agents as Market Participants

arXiv: https://arxiv.org/abs/2604.23897 · HTML: https://mlsanity.com/paper/2604.23897/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 9 of 9)

- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks](https://mlsanity.com/paper/2510.04374/) (2025)
- [Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation](https://mlsanity.com/paper/2510.11977/) (2025)
- [Towards a Science of AI Agent Reliability](https://mlsanity.com/paper/2602.16666/) (2026)
- [Calibrating the Confidence of Large Language Models by Eliciting Fidelity](https://mlsanity.com/paper/2404.02655/) (2024)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Gradientsys: A Multi-Agent LLM Scheduler with ReAct Orchestration](https://mlsanity.com/paper/2507.06520/) (2025)
- [Scaling Small Agents Through Strategy Auctions](https://mlsanity.com/paper/2602.02751/) (2026)

## Built on by (most cited first, 1 of 1)

- [Markets, Not Planners: Decentralized Orchestration of LLM Agents with Private Information](https://mlsanity.com/paper/2608.23867/) (2026)

## Abstract (arXiv)

Markets are a promising way to coordinate AI agent activity for similar reasons to those used to justify markets more broadly. In order to effectively participate in markets, agents need to have informative signals of their own ability to successfully complete a task and the cost of doing so. We propose MarketBench, a benchmark for assessing whether AI agents have these capabilities. We use a 93-task subset of SWE-bench Lite, a software engineering benchmark, with six recently released LLMs as a demonstration. These LLMs are miscalibrated on both success probability and token usage, and auctions built from these self-reports diverge from a full-information allocation. A follow-up intervention where we add information about capabilities from prior experiments to the context improves calibration, but only modestly narrows the gap to a full-information benchmark. We also document the performance of a market-based scaffolding with these LLMs. Our results point to self-assessment as a key bottleneck for market-style coordination of AI agents.