mlsanityData to 7 Oct 2026

The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learning

arXiv 2210.11111 ↗First posted 20 Oct 2022cs.LG cs.AI cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (DRL) has demonstrated impressive results in domains such as games and robotics, where task formulations are well-defined. However, few DRL benchmarks are grounded in complex, real-world environments, where safety constraints, partial observability, and the need for hand-engineered task representations pose significant challenges. To help bridge this gap, we introduce a testbed based on the pump scheduling problem in a real-world water distribution facility. The task involves controlling pumps to ensure a reliable water supply while minimizing energy consumption and respecting the constraints of the system. Our testbed includes a realistic simulator, three years of high-resolution (1-minute) operational data from human-led control, and a baseline RL task formulation. This testbed supports a wide range of research directions, including offline RL, safe exploration, inverse RL, and multi-objective optimization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 67 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

35 of 67 references.

References · 35

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2015Prioritized Experience Replaycited by 1,070
  9. 2018DeepMind Control Suitecited by 935
  10. 2017Deep Reinforcement Learning that Matterscited by 727
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
23 more
  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  3. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  4. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  5. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  6. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  7. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  8. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
  9. 2014A Survey of Multi-Objective Sequential Decision-Makingcited by 229
  10. 2018State Representation Learning for Control: An Overviewcited by 125
  11. 2020Maxmin Q-learning: Controlling the Estimation Bias of Q-learningcited by 117
  12. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  13. 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
  14. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  15. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  16. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50
  17. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
  18. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Drivingcited by 26
  19. 2020Ecological Reinforcement Learningcited by 13
  20. 2019MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actionscited by 9
  21. 2019State Representation Learning from Demonstrationcited by 5
  22. 2020Deep Reinforcement Learning for Real-Time Optimization of Pumps in Water Distribution Systemscited by 5
  23. 2021Learning State Representations via Retracing in Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WaterAdmin: Orchestrating Community Water Distribution Optimization via AI Agents
  2. 2022Launchpad: Learning to Schedule Using Offline and Online RL Methods
  3. 2022A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning
  4. 2024Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning
  5. 2020Acme: A Research Framework for Distributed Reinforcement Learning
  6. 2026Heuristic Learning for Active Flow Control Using Coding Agents
  7. 2021On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning
  8. 2024PWM: Policy Learning with Multi-Task World Models