# Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents

arXiv: https://arxiv.org/abs/2610.04040 · HTML: https://mlsanity.com/paper/2610.04040/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 6 of 6)

- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?](https://mlsanity.com/paper/2505.07078/) (2025)
- [Your AI, Not Your View: The Bias of LLMs in Investment Analysis](https://mlsanity.com/paper/2507.20957/) (2025)
- [FinResearchBench: A Logic Tree based Agent-as-a-Judge Evaluation Framework for Financial Research Agents](https://mlsanity.com/paper/2507.16248/) (2025)
- [Structured Agentic Workflows for Financial Time-Series Modeling with LLMs and Reflective Feedback](https://mlsanity.com/paper/2508.13915/) (2025)

## Abstract (arXiv)

Financial LLM agents are often evaluated by comparing their end-to-end returns with those of a baseline and testing the paired difference against zero. This measures whether deploying the agent changes realized performance, but it does not isolate event-selection skill. An agent that frequently changes positions from flat to long can earn a positive paired return from an upward-drifting event pool even when it selects events at random. We propose the Agent Policy-Value Audit, which holds fixed the observed count of each ordered action-change type and randomly reassigns them across eligible events. The average payoff from these reassignments is the composition benchmark; the difference between observed deployment value and this benchmark is selection value. In semi-synthetic benchmarks based on real earnings-event returns, a zero-centered paired test falsely attributes passive exposure to selection skill in 11.6% of no-skill replications, while the transition-matched audit reduces this rate to 5.3%. Applied retrospectively to 723 earnings events at 44 U.S. consumer-facing firms, the audit decomposes the agent's gross deployment value of +15.2 bps/event into a +25.8 composition benchmark and a -10.6 selection value. The agent does not detectably outperform matched random assignments. Financial-agent evaluations should report deployment value separately from event-selection value.