mlsanityData to 14 Oct 2026

Portfolio Reinforcement Learning with Scenario-Context Rollout

arXiv 2602.24037 ↗First posted 27 Feb 2026cs.AIcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Market regime shifts induce distribution shifts that can degrade the performance of portfolio rebalancing policies. We propose macro-conditioned scenario-context rollout (SCR) that generates plausible next-day multivariate return scenarios under stress events. However, doing so faces new challenges, as history will never tell what would have happened differently. As a result, incorporating scenario-based rewards from rollouts introduces a reward--transition mismatch in temporal-difference learning, destabilizing RL critic training. We analyze this inconsistency and show it leads to a mixed evaluation target. Guided by this analysis, we construct a counterfactual next state using the rollout-implied continuations and augment the critic agent's bootstrap target. Doing so stabilizes the learning and provides a viable bias-variance tradeoff. In out-of-sample evaluations across 31 distinct universes of U.S. equity and ETF portfolios, our method improves Sharpe ratio by up to 76% and reduces maximum drawdown by up to 53% compared with classic and RL-based portfolio rebalancing baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 43 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

11 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025FinXplore: An Adaptive Deep Reinforcement Learning Framework for Balancing and Discovering Investment Opportunities5 Sep

Other references · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  6. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  7. 2018Computational Optimal Transportcited by 780
  8. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 128
  9. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 43
  10. 2017On Generalized Bellman Equations and Temporal-Difference Learningcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Agent Regime-Conditioned Diffusion (MARCD) for CVaR-Constrained Portfolio Decisions
  2. 2026Deep Reinforcement Learning Framework for Diversified Portfolio Management Across Global Equity Markets
  3. 2025Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards
  4. 2025DeepAries: Adaptive Rebalancing Interval Selection for Enhanced Portfolio Selection
  5. 2025Diffusion-Augmented Reinforcement Learning for Robust Portfolio Optimization under Stress Scenarios
  6. 2026Addressing Market Regime Changes and Heavy-Tailed Returns in Portfolio Optimization via Bayesian VAR and Elliptical Black-Litterman
  7. 2025Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics
  8. 2025FR-LUX: Friction-Aware, Regime-Conditioned Policy Optimization for Implementable Portfolio Management