TT-DAC-PS: Twin-Target Deterministic Actor-Critic with Policy Smoothing for Optimal Trade Execution
On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance
Abstract (arXiv)
This study addresses the optimal execution of large stock sell programs by introducing TT-DAC-PS (Twin-Target Deterministic Actor-Critic with Policy Smoothing), a deterministic actor-critic architecture that combines twin exponential-moving-average critic targets with pessimistic min backup, TD3-style target policy smoothing noise, delayed actor updates, and conservative Q regularisation to curb overestimation. Exploration uses Ornstein-Uhlenbeck (OU) noise with a hybrid schedule: deterministic episode-wise decay, variance-guided adjustment based on recent reward dispersion, and a Soft Actor-Critic (SAC)-style temperature that is learned and mapped to the noise scale. The environment integrates Almgren-Chriss (AC) trade impact with Limit Order Book (LOB) prices and volumes, normalised state features, per-step volume participation caps, and a utility-based reward. The trade execution algorithm is applied to LOB data for ten U.S. stocks. Performance is assessed against reinforcement-learning baseline algorithms, including Proximal Policy Optimisation (PPO), Soft Actor-Critic (SAC), and Advantage Actor-Critic (A2C), as well as alternative trade execution algorithms, including Time-Weighted Average Price (TWAP), Volume-Weighted Average Price (VWAP), and AC. The proposed model consistently reduces mean implementation shortfall percentage with competitive variance, outperforming classical baselines and standard reinforcement-learning benchmark models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 57 references that are in the map. First posted on arXiv 7 Jun 2026.
Builds on
16 of 57 references.
Closest ancestors · 2
Same direction, newest first
- 2025Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy15 Nov
- 2024Deep Reinforcement Learning for Online Optimal Execution Strategies17 Oct
Other references · 14
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2018Exploration by Random Network Distillationcited by 993
- 2016Deep Exploration via Bootstrapped DQNcited by 527
- 2017Noisy Networks for Explorationcited by 266
- 2017Parameter Space Noise for Explorationcited by 161
- 2021Recent Advances in Reinforcement Learning in Financecited by 110
- 2018Universal features of price formation in financial markets: perspectives from Deep Learningcited by 60
- 2019ABIDES: Towards High-Fidelity Market Simulation for AI Researchcited by 32
2 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO
- 2026Plan Before You Trade: Inference-Time Optimization for RL Trading Agents
- 2023Towards Generalizable Reinforcement Learning for Trade Execution
- 2026Diverse Approaches to Optimal Execution Schedule Generation
- 2024Optimal Execution with Reinforcement Learning
- 2026Adaptive Regime-Aware Stock Price Prediction Using Autoencoder-Gated Dual Node Transformers with Reinforcement Learning Control
- 2026Reinforcement Learning for Risk-Sensitive Investment Management: a Free Energy--Entropy Duality Approach
- 2026Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining