mlsanityData to 14 Oct 2026

Smooth Imitation Learning via Smooth Costs and Smooth Policies

arXiv 2111.02354 ↗First posted 3 Nov 2021cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning (IL) is a popular approach in the continuous control setting as among other reasons it circumvents the problems of reward mis-specification and exploration in reinforcement learning (RL). In IL from demonstrations, an important challenge is to obtain agent policies that are smooth with respect to the inputs. Learning through imitation a policy that is smooth as a function of a large state-action (s-a) space (typical of high dimensional continuous control environments) can be challenging. We take a first step towards tackling this issue by using smoothness inducing regularizers on textitboth the policy and the cost models of adversarial imitation learning. Our regularizers work by ensuring that the cost function changes in a controlled manner as a function of s-a space; and the agent policy is well behaved with respect to the state space. We call our new smooth IL algorithm textitSmooth Policy and Cost Imitation Learning (SPaCIL, pronounced 'Special'). We introduce a novel metric to quantify the smoothness of the learned policies. We demonstrate SPaCIL's superior performance on continuous control tasks from MuJoCo. The algorithm not just outperforms the state-of-the-art IL algorithm on our proposed smoothness metric, but, enjoys added benefits of faster learning and substantially higher average return.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 57 references that are in the map. First posted on arXiv 3 Nov 2021.

Builds on

21 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning28 Jun

Other references · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2017Improved Training of Wasserstein GANscited by 2,235
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  8. 2016Generative Adversarial Imitation Learningcited by 1,597
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  10. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  11. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  12. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
8 more
  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  2. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  4. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 254
  5. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  6. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 118
  7. 2016Smooth Imitation Learning for Online Sequence Predictioncited by 9
  8. 2020Deep Reinforcement Learning with Robust and Smooth Policycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  2. 2026Revisiting Temporal Regularization for Smooth Control in Deep Reinforcement Learning
  3. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  4. 2021Imitation Learning by Reinforcement Learning
  5. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  6. 2025Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations
  7. 2019Reinforced Imitation in Heterogeneous Action Space
  8. 2022Robust Learning from Observation with Model Misspecification