mlsanityData to 14 Oct 2026

Realistic Market Impact Modeling for Reinforcement Learning Trading Environments

arXiv 2603.29086 ↗First posted 30 Mar 2026cs.LG cs.CEcited by 1

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Reinforcement learning (RL) has shown promise for trading, yet most open-source backtesting environments assume negligible or fixed transaction costs, causing agents to learn trading behaviors that fail under realistic execution. We introduce three Gymnasium-compatible trading environments -- MACE (Market-Adjusted Cost Execution) stock trading, margin trading, and portfolio optimization -- that integrate nonlinear market impact models grounded in the Almgren-Chriss framework and the empirically validated square-root impact law. Each environment provides pluggable cost models, permanent impact tracking with exponential decay, and comprehensive trade-level logging. We evaluate five DRL algorithms (A2C, PPO, DDPG, SAC, TD3) on the NASDAQ-100, comparing a fixed 10 bps baseline against the AC model with Optuna-tuned hyperparameters. Our results show that (i) the cost model materially changes both absolute performance and the relative ranking of algorithms across all three environments; (ii) the AC model produces dramatically different trading behavior, e.g., daily costs dropping from 200k to 8k with turnover falling from 19% to 1%; (iii) hyperparameter optimization is essential for constraining pathological trading, with costs dropping up to 82%; and (iv) algorithm-cost model interactions are strongly environment-specific, e.g., DDPG's OOS Sharpe jumps from -2.1 to 0.3 under AC in margin trading while SAC's drops from -0.5 to -1.2. We release the full suite as an open-source extension to FinRL-Meta.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 31 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

8 of 31 references.

References · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2016OpenAI Gymcited by 1,736
  8. 2021Reinforcement Learning for Quantitative Tradingcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Agentic Trading: When LLM Agents Meet Financial Marketscited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2025Right Place, Right Time: Market Simulation-based RL for Execution Optimisation
  2. 2026Can Reinforcement Learning Efficiently Discover Price Manipulation?
  3. 2025Robust Reinforcement Learning in Finance: Modeling Market Impact with Elliptic Uncertainty Sets
  4. 2023Towards Generalizable Reinforcement Learning for Trade Execution
  5. 2024Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior
  6. 2024Reinforcement Learning Framework for Quantitative Trading
  7. 2025Dueling Deep Reinforcement Learning for Financial Time Series
  8. 2026SAiFE-gym: Model-based Environments for Automated Market Making with Concentrated Liquidity