mlsanityData to 14 Oct 2026

Autoregressive Policy Optimization for Constrained Allocation Tasks

arXiv 2409.18735 ↗First posted 27 Sep 2024cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Allocation tasks represent a class of problems where a limited amount of resources must be allocated to a set of entities at each time step. Prominent examples of this task include portfolio optimization or distributing computational workloads across servers. Allocation tasks are typically bound by linear constraints describing practical requirements that have to be strictly fulfilled at all times. In portfolio optimization, for example, investors may be obligated to allocate less than 30% of the funds into a certain industrial sector in any investment period. Such constraints restrict the action space of allowed allocations in intricate ways, which makes learning a policy that avoids constraint violations difficult. In this paper, we propose a new method for constrained allocation tasks based on an autoregressive process to sequentially sample allocations for each entity. In addition, we introduce a novel de-biasing mechanism to counter the initial bias caused by sequential sampling. We demonstrate the superior performance of our approach compared to a variety of Constrained Reinforcement Learning (CRL) methods on three distinct constrained allocation tasks: portfolio optimization, computational workload distribution, and a synthetic allocation benchmark. Our code is available at: https://github.com/niklasdbs/paspo

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

25 of 31 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  6. 2018Reward Constrained Policy Optimizationcited by 417
  7. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  8. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  9. 2018Safe Exploration in Continuous Action Spacescited by 247
  10. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  11. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  12. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 92
13 more
  1. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  2. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 68
  3. 2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 62
  4. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  5. 2017Discrete Sequential Prediction of Continuous Actions for Deep RLcited by 51
  6. 2018Resource Constrained Deep Reinforcement Learningcited by 24
  7. 2022CUP: A Conservative Update Policy Algorithm for Safe Reinforcement Learningcited by 20
  8. 2017Fast MCMC sampling algorithms on polytopescited by 17
  9. 2019Solving Online Threat Screening Games using Constrained Action Space Reinforcement Learningcited by 10
  10. 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 6
  11. 2022A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learningcited by 4
  12. 2021D3PG: Dirichlet DDPG for Task Partitioning and Offloading with Constrained Hybrid Action Space in Mobile Edge Computingcited by 3
  13. 2024Simplex Decomposition for Portfolio Allocation Constraints in Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization
  2. 2025Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information
  3. 2025Optimizing Return Distributions with Distributional Dynamic Programming
  4. 2026Dynamic mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets
  5. 2025Deep Reinforcement Learning for Optimal Asset Allocation Using DDPG with TiDE
  6. 2024Improving Portfolio Optimization Results with Bandit Networks
  7. 2026EvoNash-MARL: A Closed-Loop Multi-Agent Reinforcement Learning Framework for Medium-Horizon Equity Allocation
  8. 2025SAMP-HDRL: Segmented Allocation with Momentum-Adjusted Utility for Multi-agent Portfolio Management via Hierarchical Deep Reinforcement Learning