mlsanityData to 7 Oct 2026

Universal Trading for Order Execution with Oracle Policy Distillation

arXiv 2103.10860 ↗First posted 28 Jan 2021q-fin.TR cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

As a fundamental problem in algorithmic trading, order execution aims at fulfilling a specific trading order, either liquidation or acquirement, for a given instrument. Towards effective execution strategy, recent years have witnessed the shift from the analytical view with model-based market assumptions to model-free perspective, i.e., reinforcement learning, due to its nature of sequential decision optimization. However, the noisy and yet imperfect market information that can be leveraged by the policy has made it quite challenging to build up sample efficient reinforcement learning methods to achieve effective order execution. In this paper, we propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. Particularly, this framework leverages a policy distillation method that can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information to approximate the optimal trading strategy. The extensive experiments have shown significant improvements of our method over various strong baselines, with reasonable trading actions.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 14 of its 45 references that are in the map. First posted on arXiv 28 Jan 2021.

Builds on

14 of 45 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Policy Distillationcited by 414
  4. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  5. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  6. 2020Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction Statescited by 55
  7. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  8. 2019Reinforcement Learning with Non-Markovian Rewardscited by 45
  9. 2018Double Deep Q-Learning for Optimal Executioncited by 39
  10. 2016Cryptocurrency Portfolio Management with Deep Reinforcement Learningcited by 31
  11. 2019A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Networkcited by 22
  12. 2019Distillation Strategies for Proximal Policy Optimizationcited by 7
2 more
  1. 2020Conditional Mutual information-based Contrastive Loss for Financial Time Series Forecastingcited by 5
  2. 2018Trading algorithms with learning in latent alpha modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 8; 2024: 3; 2025: 3; 2026: 3.

29 papers.

Sort:

By area:

  1. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  2. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  3. 2022Hindsight Learning for MDPs with Exogenous Inputscited by 25
  4. 2021Reinforcement Learning for Quantitative Tradingcited by 24
  5. 2022Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemblecited by 18
  6. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  7. 2021DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunitiescited by 14
  8. 2022Reinforcement Learning with Automated Auxiliary Loss Searchcited by 12
  9. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  10. 2025From Deep Learning to LLMs: A survey of AI in Quantitative Investmentcited by 9
  11. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 8
  12. 2023Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Financecited by 7
17 more
  1. 2023Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Poolscited by 7
  2. 2023IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Makingcited by 6
  3. 2023PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Marketscited by 5
  4. 2023Towards Generalizable Reinforcement Learning for Trade Executioncited by 5
  5. 2022Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Executioncited by 4
  6. 2023Microstructure-Empowered Stock Factor Extraction and Utilizationcited by 2
  7. 2024STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Tradingcited by 2
  8. 2022Safe-FinRL: A Low Bias and Variance Deep Reinforcement Learning Implementation for High-Freq Stock Tradingcited by 1
  9. 2022A Modular Framework for Reinforcement Learning Optimal Executioncited by 1
  10. 2022Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimizationcited by 1
  11. 2023Curriculum Learning and Imitation Learning for Model-free Control on Financial Time-seriescited by 1
  12. 2025QuantBench: Benchmarking AI Methods for Quantitative Investmentcited by 1
  13. 2024Large Investment Modelcited by 0
  14. 2025Two Heads are Better than One: Distilling Large Language Model Features Into Small Models with Feature Decomposition and Mixturecited by 0
  15. 2026State-Action Inpainting Diffuser for Continuous Control with Delaycited by 0
  16. 2026TT-DAC-PS: Twin-Target Deterministic Actor-Critic with Policy Smoothing for Optimal Trade Executioncited by 0
  17. 2026Can Large Language Models Execute Parent Orders?cited by 0