mlsanityData to 14 Oct 2026

Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control

arXiv 2608.09453 ↗First posted 10 Aug 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

On--off cycling is the main cause of compressor wear in residential heat pumps, yet reinforcement learning (RL) controllers for buildings typically optimise only energy cost and thermal comfort, ignoring how much the learned policy cycles. We add a levelised compressor-wear term to the control reward and study how the resulting behaviour depends on the RL algorithm. Training Soft Actor---Critic (SAC) and Proximal Policy Optimisation (PPO) on an identical Markov decision process for the BOPTEST bestest hydronic heat pump case, we find that SAC learns a continuous modulation policy that keeps the compressor permanently engaged---the operating principle of an inverter-driven heat pump---achieving zero start-ups per day, whereas PPO collapses to bang-bang control that cycles more than the baseline. On the BOPTEST emulator the SAC policy cuts thermal discomfort by up to 90.7% for an 11.5% cost increase, while eliminating all baseline cycling.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 15 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

4 of 15 references.

References · 4

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 59
  4. 2025Learning from Less: SINDy Surrogates in RLcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings
  2. 2024Constrained Reinforcement Learning for Safe Heat Pump Control
  3. 2026PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management
  4. 2025Trustworthy and Explainable Deep Reinforcement Learning for Safe and Energy-Efficient Process Control: A Use Case in Industrial Compressed Air Systems
  5. 2026Safe Deep Reinforcement Learning for Building Heating Control and Demand-side Flexibility
  6. 2026Higher-Order Action Regularization in Deep Reinforcement Learning: From Continuous Control to Building Energy Management
  7. 2026Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control
  8. 2026Quantifying the Energy Floor: Direct Measurement and Replay Buffer Bias in SAC-Based HVAC Control on sbsim