mlsanityData to 7 Oct 2026

TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution

arXiv 2104.00620 ↗First posted 16 Feb 2021q-fin.TR cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Advances in Reinforcement Learning (RL) span a wide variety of applications which motivate development in this area. While application tasks serve as suitable benchmarks for real world problems, RL is seldomly used in practical scenarios consisting of abrupt dynamics. This allows one to rethink the problem setup in light of practical challenges. We present Trade Execution using Reinforcement Learning (TradeR) which aims to address two such practical challenges of catastrophy and surprise minimization by formulating trading as a real-world hierarchical RL problem. Through this lens, TradeR makes use of hierarchical RL to execute trade bids on high frequency real market experiences comprising of abrupt price variations during the 2019 fiscal year COVID19 stock market crash. The framework utilizes an energy-based scheme in conjunction with surprise value function for estimating and minimizing surprise. In a large-scale study of 35 stock symbols from the S&P500 index, TradeR demonstrates robustness to abrupt price changes and catastrophic losses while maintaining profitable outcomes. We hope that our work serves as a motivating example for application of RL to practical problems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 16 Feb 2021.

Builds on

19 of 35 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  9. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  10. 2016The Option-Critic Architecturecited by 433
  11. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  12. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
7 more
  1. 2019Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?cited by 66
  2. 2018Practical Deep Reinforcement Learning Approach for Stock Tradingcited by 49
  3. 2020An Application of Deep Reinforcement Learning to Algorithmic Tradingcited by 30
  4. 2020A Deep Reinforcement Learning Framework for Continuous Intraday Market Biddingcited by 15
  5. 2020Application of Deep Q-Network in Portfolio Managementcited by 11
  6. 2020Deep Deterministic Portfolio Optimizationcited by 4
  7. 2020Energy-based Surprise Minimization for Multi-Agent Value Factorizationcited by 1

Built on by

Not cited yet.