mlsanityData to 14 Oct 2026

Action-Constrained Imitation Learning

arXiv 2508.14379 ↗First posted 20 Aug 2025cs.RO cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Policy learning under action constraints plays a central role in ensuring safe behaviors in various robot control and resource allocation applications. In this paper, we study a new problem setting termed Action-Constrained Imitation Learning (ACIL), where an action-constrained imitator aims to learn from a demonstrative expert with larger action space. The fundamental challenge of ACIL lies in the unavoidable mismatch of occupancy measure between the expert and the imitator caused by the action constraints. We tackle this mismatch through textittrajectory alignment and propose DTWIL, which replaces the original expert demonstrations with a surrogate dataset that follows similar state trajectories while adhering to the action constraints. Specifically, we recast trajectory alignment as a planning problem and solve it via Model Predictive Control, which aligns the surrogate trajectories with the expert trajectories based on the Dynamic Time Warping (DTW) distance. Through extensive experiments, we demonstrate that learning from the dataset generated by DTWIL significantly enhances performance across multiple robot control tasks and outperforms various benchmark imitation learning algorithms in terms of sample efficiency. Our code is publicly available at https://github.com/NYCU-RL-Bandits-Lab/ACRL-Baselines.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 39 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

33 of 39 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Diffusion Imitation from Observation7 Oct
  2. 2024Diffusion-Reward Adversarial Imitation Learning25 May
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges5 Sep
  4. 2022Imitation Learning by State-Only Distribution Matching9 Feb
  5. 2021Off-Policy Imitation Learning from Observations25 Feb
  6. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  7. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  8. 2019State Alignment-based Imitation Learning21 Nov
  9. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement10 Oct
  10. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  11. 2018Generative Adversarial Imitation from Observation17 Jul
  12. 2018Behavioral Cloning from Observation4 May

Other references · 21

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  4. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  5. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  6. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  7. 2018Safe Exploration in Continuous Action Spacescited by 247
  8. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  9. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  10. 2020Learning Off-Policy with Online Planningcited by 53
  11. 2019DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learningcited by 50
  12. 2018Experience-driven Networking: A Deep Reinforcement Learning based Approachcited by 41
9 more
  1. 2018Resource Constrained Deep Reinforcement Learningcited by 24
  2. 2019Robust Regression for Safe Exploration in Controlcited by 24
  3. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  4. 2024FlowPG: Action-constrained Policy Gradient with Normalizing Flowscited by 16
  5. 2020CFR-RL: Traffic Engineering with Reinforcement Learning in SDNcited by 12
  6. 2023A Coupled Flow Approach to Imitation Learningcited by 12
  7. 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 11
  8. 2023Generative Modelling of Stochastic Actions with Arbitrary Constraints in Reinforcement Learningcited by 10
  9. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 9

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planningcited by 3
  2. 2025Restoring Noisy Demonstration for Imitation Learning With Diffusion Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitching
  2. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
  3. 2022Differentiable Constrained Imitation Learning for Robot Motion Planning and Control
  4. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories
  5. 2024Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overview
  6. 2026Higher-Order Action Supervision Makes A Strong Policy Class
  7. 2026Implicit Drifting Policy: One-Step Action Generation via Conditional Expert Geometry
  8. 2021Bayesian Disturbance Injection: Robust Imitation Learning of Flexible Policies