mlsanityData to 7 Oct 2026

Towards Generalizable Reinforcement Learning for Trade Execution

arXiv 2307.11685 ↗First posted 12 May 2023q-fin.TR cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Optimized trade execution is to sell (or buy) a given amount of assets in a given time with the lowest possible trading cost. Recently, reinforcement learning (RL) has been applied to optimized trade execution to learn smarter policies from market data. However, we find that many existing RL methods exhibit considerable overfitting which prevents them from real deployment. In this paper, we provide an extensive study on the overfitting problem in optimized trade execution. First, we model the optimized trade execution as offline RL with dynamic context (ORDC), where the context represents market variables that cannot be influenced by the trading policy and are collected in an offline manner. Under this framework, we derive the generalization bound and find that the overfitting issue is caused by large context space and limited context samples in the offline setting. Accordingly, we propose to learn compact representations for context to address the overfitting problem, either by leveraging prior knowledge or in an end-to-end manner. To evaluate our algorithms, we also implement a carefully designed simulator based on historical limit order book (LOB) data to provide a high-fidelity benchmark for different algorithms. Our experiments on the high-fidelity simulator demonstrate that our algorithms can effectively alleviate overfitting and achieve better performance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 16 of its 53 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

16 of 53 references.

References · 16

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  3. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  4. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
  5. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  6. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  7. 2019Observational Overfitting in Reinforcement Learningcited by 90
  8. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 86
  9. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  10. 2019Get Real: Realism Metrics for Robust Limit Order Book Market Simulationscited by 40
  11. 2018Double Deep Q-Learning for Optimal Executioncited by 39
  12. 2019ABIDES: Towards High-Fidelity Market Simulation for AI Researchcited by 31
4 more
  1. 2018Variance Reduction for Reinforcement Learning in Input-Driven Environmentscited by 30
  2. 2021Universal Trading for Order Execution with Oracle Policy Distillationcited by 29
  3. 2018Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learningcited by 24
  4. 2020Auxiliary-task Based Deep Reinforcement Learning for Participant Selection Problem in Mobile Crowdsourcingcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023Pre-Trained Large Language Models for Industrial Controlcited by 15
  2. 2023A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Managementcited by 9
  3. 2024MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Tradingcited by 7
  4. 2025CTBench: Cryptocurrency Time Series Generation Benchmarkcited by 3
  5. 2024Hedging and Pricing Structured Products Featuring Multiple Underlying Assetscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization
  2. 2022Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO
  3. 2025Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learning
  4. 2025FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading
  5. 2026When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game
  6. 2022MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization
  7. 2026Learning Stock Trading Policies via Barycenter-Based Adversarial Inverse Reinforcement Learning
  8. 2025Language Model Guided Reinforcement Learning in Quantitative Trading