mlsanityData to 7 Oct 2026

DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading

arXiv 2605.25527 ↗First posted 25 May 2026cs.LG cs.CEcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper studies reinforcement learning for high-frequency trading on limit order books by pairing an Order-Flow-based state model with policy-gradient methods. Instead of value-based RL techniques like tabular Q-learning, our approach deploys policy-based methods like vanilla PPO and DeepSeekMath-inspired variants like GRPO and GSPO, that use group-normalized updates and downside-aware shaping. On backtests with financial assets AMZN, AAPL, and GOOG under a simplified backtesting setup based on spread-scaled rewards, these new policies improve net average PnL, profitability, and drawdown over the Q-Learning baseline. Our results show that (1) Order-Flow signals are an adequate state for policy RL and (2) group-aware PPO surrogates are preferable over value-based baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 6 of its 22 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

6 of 22 references.

References · 6

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Group Sequence Policy Optimizationcited by 788
  3. 2019Model-based Reinforcement Learning for Predictions and Control for Limit Order Bookscited by 14
  4. 2025Time-Constrained Recommendations: Reinforcement Learning Strategies for E-Commercecited by 3
  5. 2023Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Tradingcited by 1
  6. 2026Adaptive Estimation and Optimal Control in Offline Contextual MDPs without Stationaritycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025News-Aware Direct Reinforcement Trading for Financial Markets
  2. 2025FlowHFT: Imitation Learning via Flow Matching Policy for Optimal High-Frequency Trading under Diverse Market Conditions
  3. 2023Towards Generalizable Reinforcement Learning for Trade Execution
  4. 2022Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO
  5. 2022MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization
  6. 2026Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes
  7. 2025Language Model Guided Reinforcement Learning in Quantitative Trading
  8. 2025FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading