mlsanityData to 14 Oct 2026

OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Items

arXiv 2609.01933 ↗First posted 1 Sep 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Modern supply chain operations can require coordinating replenishment across thousands of heterogeneous items under correlated stochastic demand, heterogeneous lead times, and shared fixed ordering costs, yielding observation spaces exceeding 104 dimensions. At this scale, rolling-horizon stochastic mixed-integer linear programs (MILPs) become prohibitively slow, while standard reinforcement learning (RL) methods face increasingly challenging credit assignment in high-dimensional action spaces. We introduce OR-Transformer, a deep reinforcement learning framework for joint replenishment under stochastic demand, with an item-permutation-equivariant Transformer architecture and pathwise-gradient training through the inventory dynamics. Across problem sizes up to 1,024 inventory items, OR-Transformer increasingly outperforms learning-based and rolling-horizon MILP baselines as scale grows. It also reduces online decision-making time by over 4 million times relative to MILP solvers, enabling real-time, large-scale deep RL in supply chain operations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 27 of its 40 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

27 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026DeepStock: Reinforcement Learning with Policy Regularizations for Inventory Management20 Mar
  2. 2024VC Theory for Inventory Policies17 Apr

Other references · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Gradient Estimation Using Stochastic Computation Graphscited by 121
  5. 2015Parallelizing the dual revised simplex methodcited by 89
  6. 2024The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noisecited by 24
  7. 2022Deep Inventory Managementcited by 23
  8. 2023Efficient Policy Evaluation with Offline Data Informed Behavior Policy Designcited by 14
  9. 2026Beyond Pessimism: Offline Learning in KL-regularized Gamescited by 10
  10. 2024Efficient Multi-Policy Evaluation for Reinforcement Learningcited by 9
  11. 2024Doubly Optimal Policy Evaluation for Reinforcement Learningcited by 8
  12. 2026AI Agents for Inventory Control: Human-LLM-OR Complementaritycited by 8
13 more
  1. 2026Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewardscited by 7
  2. 2024Efficient Policy Evaluation with Safety Constraint for Reinforcement Learningcited by 6
  3. 2026Offline Two-Player Zero-Sum Markov Games with KL Regularizationcited by 6
  4. 2026Pessimism-Free Offline Learning in General-Sum Games via KL Regularizationcited by 5
  5. 2026Fast Rates in α-Potential Games via Regularized Mirror Descentcited by 5
  6. 2026Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learningcited by 5
  7. 2021Optimal Pricing of Informationcited by 4
  8. 2026Predicting Plasticity in Deep Continual Learning: A Theoretical Perspectivecited by 4
  9. 2026Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiationscited by 4
  10. 2026Convergence of Two-Timescale Markovian Stochastic Approximations with Applications in Reinforcement Learningcited by 3
  11. 2026Robust Data-Collection Policy Learning for Low-Variance Online Policy Evaluationcited by 2
  12. 2026gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methodscited by 1
  13. 2026Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradientscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Pessimistic Minimax Learning for Public-Private Information Games under Unilateral Coveragecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Deep RL Dual Sourcing Inventory Management with Supply and Capacity Risk Awareness
  2. 2025Structure-Informed Deep Reinforcement Learning for Inventory Management
  3. 2025Iterative Multi-Agent Reinforcement Learning: A Novel Approach Toward Real-World Multi-Echelon Inventory Optimization
  4. 2024Dual-Agent Deep Reinforcement Learning for Dynamic Pricing and Replenishment
  5. 2025Closing the Loop: Coordinating Inventory and Recommendation via Deep Reinforcement Learning on Multiple Timescales
  6. 2024Deep Reinforcement Learning for Dynamic Order Picking in Warehouse Operations
  7. 2025ORPR: An OR-Guided Pretrain-then-Reinforce Learning Model for Inventory Management
  8. 2025Classical and Deep Reinforcement Learning Inventory Control Policies for Pharmaceutical Supply Chains with Perishability and Non-Stationarity