# Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator

arXiv: https://arxiv.org/abs/2607.10960 · HTML: https://mlsanity.com/paper/2607.10960/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 31 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 6 of 6)

- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Double Deep Q-Learning for Optimal Execution](https://mlsanity.com/paper/1812.06600/) (2018)
- [An analysis of Uniswap markets](https://mlsanity.com/paper/1911.03380/) (2019)
- [Improved Price Oracles: Constant Function Market Makers](https://mlsanity.com/paper/2003.10001/) (2020)
- [Competition between DEXs through Dynamic Fees](https://mlsanity.com/paper/2603.09669/) (2026)
- [Optimal Fees for Liquidity Provision in Automated Market Makers](https://mlsanity.com/paper/2508.08152/) (2025)

## Abstract (arXiv)

Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs), but historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment. We therefore construct a minimal closed-loop simulator in which the missing signal exists by construction: two constant-product pools repriced by an equilibrium-inspired dynamic-fee rule, fee-sensitive noise flow, and closed-form CEX--AMM arbitrage. Equilibrium is used as a closure principle, not as an object the trader learns. Against a tuned benchmark ladder of schedule, planning, lookahead, and tabular policies, a small DQN is the only evaluated valid policy whose paired improvement over tuned one-step routing excludes zero. On a reserved final block of 1,000 seeds with completion forced to 1.0 for every policy, it reduces implementation shortfall under every tested intra-step ordering, by 13.3bps of order notional under the pre-specified agent-last ordering, and the edge is concentrated in, and learned from, dynamic-fee environments: under constant fees the paired difference is indistinguishable from zero. The result is model-conditioned counterfactual evidence about execution control in AMMs, not evidence about historical traders, equilibrium play, or deployable profit.