mlsanityData to 14 Oct 2026

Plan Before You Trade: Inference-Time Optimization for RL Trading Agents

arXiv 2605.12653 ↗First posted 12 May 2026cs.LG cs.AI stat.MLcited by 1

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for policy adjustments using price forecasts at inference time. We propose FinPILOT (Financial Plugin Inference-time Learning for Optimal Trading), a plugin inference-time policy optimization framework inspired by Model Predictive Control (MPC). Under the price-taker assumption, portfolio actions do not affect future market prices, allowing a multi-step price forecaster to provide imagined market trajectories without iterative action-conditioned rollouts. At each decision step, FinPILOT uses the forecaster's predicted price trajectory to construct an allocation-based imagined return objective and optimizes the pretrained policy at inference time before executing one step of the trade. Our framework is compatible with any pretrained policy and adapts the policy using the forecaster's predictions without modifying the original RL training pipeline. Evaluated with five pretrained policies (PPO, SAC, A2C, TD3, DDPG) across DJ30 equities and foreign exchange (FX) over three rolling held-out test years (30 settings in total), FinPILOT improves Total Return in 28 of 30 settings and Sharpe ratio in 27 of 30, with gains spanning both stochastic and deterministic policies. Direct single- and multi-period portfolio optimizers supplied with the same forecasts do not reproduce these aggregate gains under our evaluation protocol. Further, using controlled forecasts at calibrated quality levels, we show that both the magnitude and consistency of FinPILOT's performance increase as forecast error is reduced.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 31 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

18 of 31 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  10. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  11. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  12. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
6 more
  1. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 128
  2. 2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 87
  3. 2020Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction Statescited by 55
  4. 2017Multi-Period Trading via Convex Optimizationcited by 21
  5. 2021DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunitiescited by 15
  6. 2026Inference Time Policy Optimization for Offline RL with Differentiable World Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Agentic Trading: When LLM Agents Meet Financial Marketscited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards Generalizable Reinforcement Learning for Trade Execution
  2. 2026Can Reinforcement Learning Efficiently Discover Price Manipulation?
  3. 2026When a Correct Reward Is Not Enough: Diagnosing and Guiding PPO in an Analytically Solved Broker-Trader Game
  4. 2025Deep reinforcement learning for optimal trading with partial information
  5. 2025Right Place, Right Time: Market Simulation-based RL for Execution Optimisation
  6. 2025Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization
  7. 2025Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards
  8. 2022MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization