mlsanityData to 7 Oct 2026

Soft Actor-Critic With Integer Actions

arXiv 2109.08512 ↗First posted 17 Sep 2021cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is well-studied under discrete actions. Integer actions setting is popular in the industry yet still challenging due to its high dimensionality. To this end, we study reinforcement learning under integer actions by incorporating the Soft Actor-Critic (SAC) algorithm with an integer reparameterization. Our key observation for integer actions is that their discrete structure can be simplified using their comparability property. Hence, the proposed integer reparameterization does not need one-hot encoding and is of low dimensionality. Experiments show that the proposed SAC under integer actions is as good as the continuous action version on robot control tasks and outperforms Proximal Policy Optimization on power distribution systems control tasks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

27 of 37 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2016OpenAI Gymcited by 1,684
  10. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  11. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  12. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
15 more
  1. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  2. 2014A* Samplingcited by 150
  3. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  4. 2017Action Branching Architectures for Deep Reinforcement Learningcited by 90
  5. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 65
  6. 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 65
  7. 2021Learning to run a Power Network Challenge: a Retrospective Analysiscited by 52
  8. 2017Discrete Sequential Prediction of Continuous Actions for Deep RLcited by 50
  9. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 45
  10. 2020Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimatorcited by 37
  11. 2019Discrete and Continuous Action Representation for Practical RL in Video Gamescited by 34
  12. 2018Boosting Trust Region Policy Optimization by Normalizing Flows Policycited by 11
  13. 2021PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systemscited by 10
  14. 2021Discrete-continuous Action Space Policy Gradient-based Attention for Image-Text Matchingcited by 8
  15. 2021Behavior-Guided Actor-Critic: Improving Exploration via Learning Policy Behavior Representation for Deep Reinforcement Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Training Discrete Deep Generative Models via Gapped Straight-Through Estimatorcited by 8
  2. 2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3