# Reinforcement Learning with Expert Trajectory For Quantitative Trading

arXiv: https://arxiv.org/abs/2105.03844 · HTML: https://mlsanity.com/paper/2105.03844/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 26 references that are in the map.
- First posted on arXiv 9 May 2021.

## Builds on (most cited first, 2 of 2)

- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Risk-Averse Trust Region Optimization for Reward-Volatility Reduction](https://mlsanity.com/paper/1912.03193/) (2019)

## Built on by (most cited first, 3 of 3)

- [QADQN: Quantum Attention Deep Q-Network for Financial Market Prediction](https://mlsanity.com/paper/2408.03088/) (2024)
- [High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning](https://mlsanity.com/paper/2112.04755/) (2021)
- [QTMRL: An Agent for Quantitative Trading Decision-Making Based on Multi-Indicator Guided Reinforcement Learning](https://mlsanity.com/paper/2508.20467/) (2025)

## Abstract (arXiv)

In recent years, quantitative investment methods combined with artificial intelligence have attracted more and more attention from investors and researchers. Existing related methods based on the supervised learning are not very suitable for learning problems with long-term goals and delayed rewards in real futures trading. In this paper, therefore, we model the price prediction problem as a Markov decision process (MDP), and optimize it by reinforcement learning with expert trajectory. In the proposed method, we employ more than 100 short-term alpha factors instead of price, volume and several technical factors in used existing methods to describe the states of MDP. Furthermore, unlike DQN (deep Q-learning) and BC (behavior cloning) in related methods, we introduce expert experience in training stage, and consider both the expert-environment interaction and the agent-environment interaction to design the temporal difference error so that the agents are more adaptable for inevitable noise in financial data. Experimental results evaluated on share price index futures in China, including IF (CSI 300) and IC (CSI 500), show that the advantages of the proposed method compared with three typical technical analysis and two deep leaning based methods.