mlsanityData to 7 Oct 2026

Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programming

arXiv 2202.13436 ↗First posted 27 Feb 2022cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a framework, called neural-progressive hedging (NP), that leverages stochastic programming during the online phase of executing a reinforcement learning (RL) policy. The goal is to ensure feasibility with respect to constraints and risk-based objectives such as conditional value-at-risk (CVaR) during the execution of the policy, using probabilistic models of the state transitions to guide policy adjustments. The framework is particularly amenable to the class of sequential resource allocation problems since feasibility with respect to typical resource constraints cannot be enforced in a scalable manner. The NP framework provides an alternative that adds modest overhead during the online phase. Experimental results demonstrate the efficacy of the NP framework on two continuous real-world tasks: (i) the portfolio optimization problem with liquidity constraints for financial planning, characterized by non-stationary state distributions; and (ii) the dynamic repositioning problem in bike sharing systems, that embodies the class of supply-demand matching problems. We show that the NP framework produces policies that are better than deep RL and other baseline approaches, adapting to non-stationarity, whilst satisfying structural constraints and accommodating risk measures in the resulting policies. Additional benefits of the NP framework are ease of implementation and better explainability of the policies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 27 Feb 2022.

Builds on

24 of 47 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2017Constrained Policy Optimizationcited by 928
  7. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  8. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  9. 2020Projection-Based Constrained Policy Optimizationcited by 195
  10. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  11. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  12. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
12 more
  1. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  2. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  3. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  4. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  5. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  6. 2017RLlib: Abstractions for Distributed Reinforcement Learningcited by 53
  7. 2018Fast Policy Learning through Imitation and Reinforcementcited by 26
  8. 2014Robust Sample Average Approximationcited by 25
  9. 2016PLATO: Policy Learning using Adaptive Trajectory Optimizationcited by 23
  10. 2018Resource Constrained Deep Reinforcement Learningcited by 23
  11. 2016Learning from the Hindsight Plan -- Episodic MPC Improvementcited by 15
  12. 2019Adaptive Online Planning for Continual Lifelong Learningcited by 7

Built on by

Not cited yet.