Knowledge or Calculator? Decomposing the Skill Premium in Verifiable Financial Agent Workflows
Not on the map. Why?
Abstract (arXiv)
Financial AI agents must do more than retrieve facts: investment workflows require correct quantitative execution, reliable use of procedural resources, and auditable structured outputs. We introduce FinSkillBench, an evaluation suite of 2,603 point in time episodes across 12 subtasks in portfolio construction, risk management, and fundamental analysis, with hidden regenerable ground truth and task specific deterministic verifiers. Executing 17,820 episodes across 9 models and 3 resource conditions, the paired analysis across 8 models shows that curated skill packages raise mean scores by +16.2 points (0.366 to 0.528), whereas skills generated within a single episode add only +0.5 points while consuming more tokens and turns. We then decompose the curated premium by granting human authored procedural documents and executable domain tools separately: documents alone add +5.6 points, tools alone add +19.5 points, and their combination is subadditive. The premium is strongly workflow dependent: executable tools dominate numerically intensive workflows, documentation matters more when procedural or output schema guidance is the bottleneck, and interpretive tasks benefit from both. The effects are sign stable across 10 scoring variants and cluster bootstrap analyses, and an independently implemented second harness reproduces the directional pattern while showing that effect magnitudes depend on how tools and data are exposed. Overall, a measured "skill premium" is a property of the full model, resource, and harness system rather than of the underlying model alone.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 34 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
13 of 34 references.
References · 13
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
- 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
- 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 40
- 2025FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluationcited by 30
- 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challengingcited by 28
- 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 10
- 2026FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Usecited by 10
- 2024Breaking Down Financial News Impact: A Novel AI Approach with Geometric Hypergraphscited by 6
- 2025RicciFlowRec: A Geometric Root Cause Recommender Using Ricci Curvature on Financial Graphscited by 2
- 2025From News to Returns: A Granger-Causal Hypergraph Transformer on the Spherecited by 2
Built on by
Not cited yet.