# ViperQ: Order Flow Pattern Recognition via Auction Market Theory for Reinforcement Learning Trading

arXiv: https://arxiv.org/abs/2609.13825 · HTML: https://mlsanity.com/paper/2609.13825/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 12 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [An Application of Deep Reinforcement Learning to Algorithmic Trading](https://mlsanity.com/paper/2004.06627/) (2020)
- [Deep Reinforcement Learning for Active High Frequency Trading](https://mlsanity.com/paper/2101.07107/) (2021)
- [Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy](https://mlsanity.com/paper/2511.12120/) (2025)
- [Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness](https://mlsanity.com/paper/2306.12806/) (2023)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution](https://mlsanity.com/paper/2511.15262/) (2025)
- [Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics](https://mlsanity.com/paper/2509.12456/) (2025)

## Abstract (arXiv)

Reinforcement learning trading systems published in the academic literature overwhelmingly rely on price-aggregate state representations (OHLCV bars) or limit-order-book depth features, leaving microstructure pattern theories from the practitioner literature, namely Auction Market Theory and Market Profile, without a peer-reviewed computational instantiation. We present ViperQ, a reinforcement learning system whose state representation is built explicitly from Auction Market Theory primitives: Volume Point of Control, Value Area position, Low Volume Node flags, Cumulative Volume Delta divergence, and tape-velocity signatures, assembled into a 20-dimensional Z-normalised vector. Two Proximal Policy Optimisation agents are trained with a prospect theory-grounded asymmetric reward function that penalises losing holds at a magnitude consistent with Kahneman and Tversky's loss-aversion coefficient. Evaluated on a held-out twelve-month partition of institutional tick data the agents have never seen, ViperQ achieves +163.6% ROI on TSLA (-27.5% max drawdown, 27,019 trades) and +116.5% ROI on NVDA (-47.8% max drawdown, 12,892 trades) under zero leverage. The results establish Auction Market Theory features as a tractable structured input modality for sequential decision-making on financial time series and motivate further work on microstructure-aware policy learning.