# Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems

arXiv: https://arxiv.org/abs/2606.08285 · HTML: https://mlsanity.com/paper/2606.08285/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 34 of its 77 references that are in the map.
- First posted on arXiv 6 Jun 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 30 of 34)

- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [FinGPT: Open-Source Financial Large Language Models](https://mlsanity.com/paper/2306.06031/) (2023)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models](https://mlsanity.com/paper/2304.07619/) (2023)
- [FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making](https://mlsanity.com/paper/2407.06567/) (2024)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent](https://mlsanity.com/paper/2412.18174/) (2024)
- [Large Language Model Agent in Financial Trading: A Survey](https://mlsanity.com/paper/2408.06361/) (2024)
- [Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment](https://mlsanity.com/paper/2308.00016/) (2023)
- [AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay](https://mlsanity.com/paper/2502.16789/) (2025)
- [FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models](https://mlsanity.com/paper/2405.14767/) (2024)
- [Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models](https://mlsanity.com/paper/2402.03659/) (2024)
- [Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications](https://mlsanity.com/paper/2408.11878/) (2024)
- [QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model](https://mlsanity.com/paper/2402.03755/) (2024)
- [When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments](https://mlsanity.com/paper/2407.18957/) (2024)
- [AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets](https://mlsanity.com/paper/2512.10971/) (2025)
- [FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications](https://mlsanity.com/paper/2403.12285/) (2024)
- [AlphaAgents: Large Language Model based Multi-Agents for Equity Portfolio Constructions](https://mlsanity.com/paper/2508.11152/) (2025)
- [Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance](https://mlsanity.com/paper/2601.13770/) (2026)
- [StockGPT: A GenAI Model for Stock Prediction and Trading](https://mlsanity.com/paper/2404.05101/) (2024)
- [ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism](https://mlsanity.com/paper/2508.00554/) (2025)
- [Sentiment trading with large language models](https://mlsanity.com/paper/2412.19245/) (2024)
- [Toward Expert Investment Teams:A Multi-Agent LLM System with Fine-Grained Trading Tasks](https://mlsanity.com/paper/2602.23330/) (2026)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [Hubble: An LLM-Driven Agentic Framework for Safe, Diverse, and Reproducible Alpha Factor Discovery](https://mlsanity.com/paper/2604.09601/) (2026)

## Abstract (arXiv)

Execution realism remains weakly standardized in research on large-language-model-based trading systems, limiting comparison and reproduction across backtests, simulations, and portfolio benchmarks. This article presents a PRISMA-ScR systematic scoping review with a nested execution-reproducibility evidence audit. Eight reproducible query families returned 796 query-level records; DOI/title deduplication left 687 unique records, 101 reports were sought for retrieval, and 59 full texts were recovered through direct and fallback open-access routes. Fifty-three reports are provisionally eligible for expanded evidence charting, subject to reconciliation of two blind author-validation packets. The charting framework separately evaluates point-in-time controls, temporal splits, held-out evaluation, trading costs and turnover, execution semantics, universe construction, architecture reporting, and artifact availability. Direct-trading, portfolio or alpha-construction, and benchmark studies are retained as distinct descriptive subgroups rather than pooled as a common performance estimand. Search responses, retrieval attempts, file hashes, evidence excerpts, screening decisions, and coding worksheets are archived with the review materials. Final field-level counts are intentionally withheld until author validation is complete.