mlsanityData to 7 Oct 2026

Learning to Generate All Feasible Actions

arXiv 2301.11461 ↗First posted 26 Jan 2023cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Modern cyber-physical systems are becoming increasingly complex to model, thus motivating data-driven techniques such as reinforcement learning (RL) to find appropriate control agents. However, most systems are subject to hard constraints such as safety or operational bounds. Typically, to learn to satisfy these constraints, the agent must violate them systematically, which is computationally prohibitive in most systems. Recent efforts aim to utilize feasibility models that assess whether a proposed action is feasible to avoid applying the agent's infeasible action proposals to the system. However, these efforts focus on guaranteeing constraint satisfaction rather than the agent's learning efficiency. To improve the learning process, we introduce action mapping, a novel approach that divides the learning process into two steps: first learn feasibility and subsequently, the objective by mapping actions into the sets of feasible actions. This paper focuses on the feasibility part by learning to generate all feasible actions through self-supervised querying of the feasibility model. We train the agent by formulating the problem as a distribution matching problem and deriving gradient estimators for different divergences. Through an illustrative example, a robotic path planning scenario, and a robotic grasping simulation, we demonstrate the agent's proficiency in generating actions across disconnected feasible action sets. By addressing the feasibility step, this paper makes it possible to focus future work on the objective part of action mapping, paving the way for an RL framework that is both safe and efficient.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 69 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

33 of 69 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  8. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  9. 2017Safe Reinforcement Learning via Shieldingcited by 427
  10. 2019Control Barrier Functions: Theory and Applicationscited by 332
  11. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  12. 2018Safe Exploration in Continuous Action Spacescited by 237
21 more
  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  2. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
  3. 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 195
  4. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  5. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
  6. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  7. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 131
  8. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  9. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  10. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  11. 2020Conservative Safety Critics for Explorationcited by 107
  12. 2021Adapting to Misspecification in Contextual Banditscited by 81
  13. 2019Antipodal Robotic Grasping using Generative Residual Convolutional Neural Networkcited by 59
  14. 2020The Expressive Power of a Class of Normalizing Flow Modelscited by 35
  15. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 32
  16. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  17. 2019Semi-Parametric Efficient Policy Learning with Continuous Actionscited by 26
  18. 2020Representational aspects of depth and conditioning in normalizing flowscited by 25
  19. 2020Efficient Contextual Bandits with Continuous Actionscited by 24
  20. 2022Contextual Bandits with Smooth Regret: Efficient Learning in Continuous Action Spacescited by 17
  21. 2021Safe-visor Architecture for Sandboxing (AI-based) Unverified Controllers in Stochastic Cyber-Physical Systemscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Improving Stochastic Action-Constrained Reinforcement Learning via Truncated Distributionscited by 2
  2. 2025Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learningcited by 0