FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?
Not on the map. Why?
Abstract (arXiv)
Financial scenarios are diverse and complex, spanning varying data conditions, tool configurations, and workflows. Yet existing CUA, Computer-Using Agent, evaluation tasks remain largely manually constructed, limiting scalable coverage of real-world financial scenarios. Then, can agents autonomously construct diverse CUA evaluation tasks for financial scenarios? Evaluating this capability poses three key challenges: scenario coverage of construction requests, fair comparison across construction methods, and reliable assessment of generated task quality. To solve these, we introduce FinCUABuildBench, a benchmark for evaluating financial CUA task construction, featuring: (i) 576 construction requests covering 24 financial workflows and three types of runtime variation; (ii) standardized input, budget, and output specifications; and (iii) a task qualification mechanism based on execution tests and quality checks. We further introduce FinCUABuildAgent, a multi-agent system for automatically constructing dynamic financial CUA evaluation tasks. It consists of three modules that jointly construct tasks, environments, and validators. On FinCUABuildBench, under the same model backbone, existing agent-based construction methods achieve strict qualification rates of only 1.3-8.3%, while FinCUABuildAgent reaches 31.3%. Downstream evaluations further show that the constructed tasks can effectively differentiate CUA task-execution capabilities. These results demonstrate that agents can autonomously construct financial CUA tasks with meaningful evaluation value, offering a practical path toward broader evaluation coverage in financial scenarios. Code: https://github.com/FengxianJi/FinCUABuild
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
22 of 30 references.
References · 22
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
- 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
- 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 121
- 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 111
- 2023Skill-Mix: a Flexible and Expandable Family of Evaluations for AI modelscited by 64
10 more
- 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 63
- 2024Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?cited by 56
- 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 46
- 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 40
- 2026FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answeringcited by 16
- 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 10
- 2026FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocolcited by 8
- 2026BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agentscited by 7
- 2026Benchmark Everything Everywhere All at Oncecited by 6
- 2026Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generationcited by 2
Built on by
Not cited yet.