# RAmmStein: Regime Adaptation in Mean-reverting Markets with Stein Thresholds -- Optimal Impulse Control in Concentrated AMMs

arXiv: https://arxiv.org/abs/2602.19419 · HTML: https://mlsanity.com/paper/2602.19419/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 13 of 13)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [Market Making via Reinforcement Learning](https://mlsanity.com/paper/1804.04216/) (2018)
- [Double Deep Q-Learning for Optimal Execution](https://mlsanity.com/paper/1812.06600/) (2018)
- [Strategic Liquidity Provision in Uniswap v3](https://mlsanity.com/paper/2106.12033/) (2021)
- [Automated Market Making and Loss-Versus-Rebalancing](https://mlsanity.com/paper/2208.06046/) (2022)
- [Adaptive Liquidity Provision in Uniswap V3 with Deep Reinforcement Learning](https://mlsanity.com/paper/2309.10129/) (2023)
- [Uniswap Liquidity Provision: An Online Learning Approach](https://mlsanity.com/paper/2302.00610/) (2023)
- [A Myersonian Framework for Optimal Liquidity Provision in Automated Market Makers](https://mlsanity.com/paper/2303.00208/) (2023)
- [An Empirical Study of Market Inefficiencies in Uniswap and SushiSwap](https://mlsanity.com/paper/2203.07774/) (2022)
- [Improving DeFi Accessibility through Efficient Liquidity Provisioning with Deep Reinforcement Learning](https://mlsanity.com/paper/2501.07508/) (2025)
- [Efficient Rollup Batch Posting Strategy on Base Layer](https://mlsanity.com/paper/2212.10337/) (2022)

## Abstract (arXiv)

Concentrated liquidity provision in decentralized exchanges presents a fundamental Impulse Control problem. Liquidity Providers (LPs) face a non-trivial trade-off between maximizing fee accrual through tight price-range concentration and minimizing the friction costs of rebalancing, including gas fees and swap slippage. Existing methods typically employ heuristic or threshold strategies that fail to account for market dynamics. This paper formulates liquidity management as an optimal control problem and derives the corresponding Hamilton-Jacobi-Bellman quasi-variational inequality (HJB-QVI). We present an approximate solution RAmmStein, a Deep Reinforcement Learning method that incorporates the mean-reversion speed (theta) of an Ornstein-Uhlenbeck process among other features as input to the model. We demonstrate that the agent learns to separate the state space into regions of action and inaction. We further extend the framework with RAmmStein-Width, which jointly optimizes rebalancing timing and position width via a 6-action DDQN. We evaluate the framework using high-frequency 1Hz Coinbase trade data comprising over 6.8M trades on a realistic environment (10M TVL, 1% default width). Experimental results show that RAmmStein achieves a net ROI of 1.60%, the highest among all realistic (non-omniscient) strategies, while greedy strategies lose up to -8.4% to gas costs. Notably, the agent reduces rebalancing frequency by 85% compared to greedy rebalancing. RAmmStein-Width discovers extreme parsimony on its own, executing only 9 rebalances and 40 in gas, and degrades more slowly than all active strategies at elevated gas costs. Our results demonstrate that regime-aware laziness can significantly improve capital efficiency by preserving the returns that would otherwise be eroded by the operational costs.