mlsanityData to 7 Oct 2026

Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Management

arXiv 2305.01461 ↗First posted 2 May 2023eess.SY cs.AI cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

Many optimal control problems require the simultaneous output of discrete and continuous control variables. These problems are usually formulated as mixed-integer optimal control (MIOC) problems, which are challenging to solve due to the complexity of the solution space. Numerical methods such as branch-and-bound are computationally expensive and undesirable for real-time control. This paper proposes a novel hybrid-action reinforcement learning (HARL) algorithm, twin delayed deep deterministic actor-Q (TD3AQ), for MIOC problems. TD3AQ combines the advantages of both actor-critic and Q-learning methods, and can handle the discrete and continuous action spaces simultaneously. The proposed algorithm is evaluated on a plug-in hybrid electric vehicle (PHEV) energy management problem, where real-time control of the discrete variables, clutch engagement/disengagement and gear shift, and continuous variable, engine torque, is essential to maximize fuel economy while satisfying driving constraints. Simulation outcomes demonstrate that TD3AQ achieves control results close to optimality when compared with dynamic programming (DP), with just 4.69% difference. Furthermore, it surpasses the performance of baseline reinforcement learning algorithms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 48 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

14 of 48 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  6. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 65
  7. 2015Reinforcement Learning with Parameterized Actionscited by 61
  8. 2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 54
  9. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 53
  10. 2020Continuous-Discrete Reinforcement Learning for Hybrid Control in Roboticscited by 42
  11. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 33
  12. 2019Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spacescited by 22
2 more
  1. 2021Homotopy Continuation Enhanced Branch and Bound Algorithms for Strongly Nonconvex Mixed-Integer Nonlinear Programming Problemscited by 1
  2. 2022Tailored Presolve Techniques in Branch-and-Bound Method for Fast Mixed-Integer Optimal Control Applicationscited by 1

Built on by

Not cited yet.