mlsanityData to 14 Oct 2026

Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator

arXiv 2607.10960 ↗First posted 12 Jul 2026cs.LG q-fin.CP stat.MLcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs), but historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment. We therefore construct a minimal closed-loop simulator in which the missing signal exists by construction: two constant-product pools repriced by an equilibrium-inspired dynamic-fee rule, fee-sensitive noise flow, and closed-form CEX--AMM arbitrage. Equilibrium is used as a closure principle, not as an object the trader learns. Against a tuned benchmark ladder of schedule, planning, lookahead, and tabular policies, a small DQN is the only evaluated valid policy whose paired improvement over tuned one-step routing excludes zero. On a reserved final block of 1,000 seeds with completion forced to 1.0 for every policy, it reduces implementation shortfall under every tested intra-step ordering, by 13.3bps of order notional under the pre-specified agent-last ordering, and the edge is concentrated in, and learned from, dynamic-fee environments: under constant fees the paired difference is indistinguishable from zero. The result is model-conditioned counterfactual evidence about execution control in AMMs, not evidence about historical traders, equilibrium play, or deployable profit.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 31 references that are in the map. First posted on arXiv 12 Jul 2026.

Builds on

6 of 31 references.

References · 6

  1. 2017Deep Reinforcement Learning that Matterscited by 754
  2. 2018Double Deep Q-Learning for Optimal Executioncited by 43
  3. 2019An analysis of Uniswap marketscited by 12
  4. 2020Improved Price Oracles: Constant Function Market Makerscited by 12
  5. 2026Competition between DEXs through Dynamic Feescited by 3
  6. 2025Optimal Fees for Liquidity Provision in Automated Market Makerscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SAiFE-gym: Model-based Environments for Automated Market Making with Concentrated Liquidity
  2. 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution
  3. 2026Concentrated Liquidity Provision: a Reinforcement Learning Perspective
  4. 2025Resolving Latency and Inventory Risk in Market Making with Reinforcement Learning
  5. 2025Right Place, Right Time: Market Simulation-based RL for Execution Optimisation
  6. 2024Optimal Execution with Reinforcement Learning
  7. 2025Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Audits
  8. 2025RL-Exec: Impact-Aware Reinforcement Learning for Opportunistic Optimal Liquidation, Outperforms TWAP and a Book-Liquidity VWAP on BTC-USD Replays