mlsanityData to 7 Oct 2026

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

arXiv 2109.05490 ↗First posted 12 Sep 2021cs.LG cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Discrete-continuous hybrid action space is a natural setting in many practical problems, such as robot control and game AI. However, most previous Reinforcement Learning (RL) works only demonstrate the success in controlling with either discrete or continuous action space, while seldom take into account the hybrid action space. One naive way to address hybrid action RL is to convert the hybrid action space into a unified homogeneous action space by discretization or continualization, so that conventional RL algorithms can be applied. However, this ignores the underlying structure of hybrid action space and also induces the scalability issue and additional approximation difficulties, thus leading to degenerated results. In this paper, we propose Hybrid Action Representation (HyAR) to learn a compact and decodable latent representation space for the original hybrid action space. HyAR constructs the latent space and embeds the dependence between discrete action and continuous parameter via an embedding table and conditional Variantional Auto-Encoder (VAE). To further improve the effectiveness, the action representation is trained to be semantically smooth through unsupervised environmental dynamics prediction. Finally, the agent then learns its policy with conventional DRL algorithms in the learned representation space and interacts with the environment by decoding the hybrid action embeddings to the original action space. We evaluate HyAR in a variety of environments with discrete-continuous action space. The results demonstrate the superiority of HyAR when compared with previous baselines, especially for high-dimensional action spaces.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 12 Sep 2021.

Builds on

25 of 46 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  7. 2021Reinforcement Learning with Prototypical Representationscited by 179
  8. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  9. 2019Learning Action Representations for Reinforcement Learningcited by 75
  10. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 65
  11. 2015Reinforcement Learning with Parameterized Actionscited by 61
  12. 2018Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Spacecited by 54
13 more
  1. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  2. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 53
  3. 2021Improving black-box optimization in VAE latent space using decoder uncertaintycited by 51
  4. 2020Continuous-Discrete Reinforcement Learning for Hybrid Control in Roboticscited by 42
  5. 2021Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Predictioncited by 36
  6. 2019Dynamics-aware Embeddingscited by 34
  7. 2019Discrete and Continuous Action Representation for Practical RL in Video Gamescited by 34
  8. 2019Multi-Pass Q-Networks for Deep Reinforcement Learning with Parameterised Action Spacescited by 22
  9. 2021Automatic Web Testing using Curiosity-Driven Reinforcement Learningcited by 19
  10. 2018Hierarchical Approaches for Reinforcement Learning in Parameterized Action Spacecited by 15
  11. 2019Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spacescited by 15
  12. 2020Continuous Multiagent Control using Collective Behavior Entropy for Large-Scale Home Energy Managementcited by 7
  13. 2021Foresee then Evaluate: Decomposing Value Estimation with Latent Future Predictioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 5; 2024: 7; 2025: 10; 2026: 6.

33 papers.

Sort:

By area:

  1. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  2. 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 53
  3. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  4. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  5. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  6. 2024Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churncited by 16
  7. 2024Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Casescited by 12
  8. 2023Challenges for Reinforcement Learning in Quantum Circuit Designcited by 9
  9. 2022Meta-Learning Parameterized Skillscited by 7
  10. 2024Model-based Reinforcement Learning for Parameterized Action Spacescited by 7
  11. 2025The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learningcited by 6
  12. 2022Reinforcement learning for traffic signal control in hybrid action spacecited by 5
21 more
  1. 2024Action Mapping for Reinforcement Learning in Continuous Environments with Constraintscited by 4
  2. 2022Distributional Reinforcement Learning for Scheduling of Chemical Production Processescited by 2
  3. 2024CycLight: learning traffic signal cooperation with a cycle-level strategycited by 1
  4. 2025Action Space Reduction Strategies for Reinforcement Learning in Autonomous Drivingcited by 1
  5. 2025Is Exploration or Optimization the Problem for Deep Reinforcement Learning?cited by 1
  6. 2025Logic-informed reinforcement learning for cross-domain optimization of large-scale cyber-physical systemscited by 1
  7. 2026Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradientscited by 1
  8. 2023Mixed-Integer Optimal Control via Reinforcement Learning: A Case Study on Hybrid Electric Vehicle Energy Managementcited by 0
  9. 2023Maneuver Decision-Making Through Proximal Policy Optimization And Monte Carlo Tree Searchcited by 0
  10. 2024Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learningcited by 0
  11. 2025Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processescited by 0
  12. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 0
  13. 2025Pose-RFT: Enhancing MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuningcited by 0
  14. 2025Strategic Communication under Threat: Learning Information Trade-offs in Pursuit-Evasion Gamescited by 0
  15. 2025Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agentscited by 0
  16. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actionscited by 0
  17. 2026Breaking the Grid: Distance-Guided Reinforcement Learning in Large Discrete Action Spacescited by 0
  18. 2026Temporal Action Representation Learning for Tactical Resource Control and Subsequent Maneuver Generationcited by 0
  19. 2026Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Setscited by 0
  20. 2026HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Modelscited by 0
  21. 2026Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processescited by 0