ViperQ: Order Flow Pattern Recognition via Auction Market Theory for Reinforcement Learning Trading
On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance
Abstract (arXiv)
Reinforcement learning trading systems published in the academic literature overwhelmingly rely on price-aggregate state representations (OHLCV bars) or limit-order-book depth features, leaving microstructure pattern theories from the practitioner literature, namely Auction Market Theory and Market Profile, without a peer-reviewed computational instantiation. We present ViperQ, a reinforcement learning system whose state representation is built explicitly from Auction Market Theory primitives: Volume Point of Control, Value Area position, Low Volume Node flags, Cumulative Volume Delta divergence, and tape-velocity signatures, assembled into a 20-dimensional Z-normalised vector. Two Proximal Policy Optimisation agents are trained with a prospect theory-grounded asymmetric reward function that penalises losing holds at a magnitude consistent with Kahneman and Tversky's loss-aversion coefficient. Evaluated on a held-out twelve-month partition of institutional tick data the agents have never seen, ViperQ achieves +163.6% ROI on TSLA (-27.5% max drawdown, 27,019 trades) and +116.5% ROI on NVDA (-47.8% max drawdown, 12,892 trades) under zero leverage. The results establish Auction Market Theory features as a tractable structured input modality for sequential decision-making on financial time series and motivate further work on microstructure-aware policy learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 33 references that are in the map. First posted on arXiv 12 Sep 2026.
Builds on
11 of 33 references.
Closest ancestors · 4
Same direction, newest first
- 2025Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution19 Nov
- 2025Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy15 Nov
- 2025Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics15 Sep
- 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Survey20 Aug
Other references · 7
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Recent Advances in Reinforcement Learning in Financecited by 110
- 2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 87
- 2019Deep Reinforcement Learning for Tradingcited by 73
- 2020An Application of Deep Reinforcement Learning to Algorithmic Tradingcited by 30
- 2021Deep Reinforcement Learning for Active High Frequency Tradingcited by 21
- 2023Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustnesscited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reinforcement Learning with Hawkes-Structured Networks for Market Making
- 2023IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making
- 2026SAiFE-gym: Model-based Environments for Automated Market Making with Concentrated Liquidity
- 2025FlowHFT: Imitation Learning via Flow Matching Policy for Optimal High-Frequency Trading under Diverse Market Conditions
- 2026DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading
- 2025Right Place, Right Time: Market Simulation-based RL for Execution Optimisation
- 2024Exploiting Risk-Aversion and Size-dependent fees in FX Trading with Fitted Natural Actor-Critic
- 2024Deviations from the Nash equilibrium in a two-player optimal execution game with reinforcement learning