mlsanityData to 14 Oct 2026

Unlocking the Potential of Soft Actor-Critic for Imitation Learning

arXiv 2509.24539 ↗First posted 29 Sep 2025cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

also in 1 more
Abstract (arXiv)

Learning-based methods have enabled robots to acquire bio-inspired movements with increasing levels of naturalness and adaptability. Among these, Imitation Learning (IL) has proven effective in transferring complex motion patterns from animals to robotic systems. However, current state-of-the-art frameworks predominantly rely on Proximal Policy Optimization (PPO), an on-policy algorithm that prioritizes stability over sample efficiency and policy generalization. This paper proposes a novel IL framework that combines Adversarial Motion Priors (AMP) with the off-policy Soft Actor-Critic (SAC) algorithm to overcome these limitations. This integration leverages replay-driven learning and entropy-regularized exploration, enabling naturalistic behavior and task execution, improving data efficiency and robustness. We evaluate the proposed approach (AMP+SAC) on quadruped gaits involving multiple reference motions and diverse terrains. Experimental results demonstrate that the proposed framework not only maintains stable task execution but also achieves higher imitation rewards compared to the widely used AMP+PPO method. These findings highlight the potential of an off-policy IL formulation for advancing motion generation in robotics.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 24 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

14 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions16 Sep
  2. 2021Imitation Learning from MPC for Quadrupedal Multi-Gait Control26 Mar

Other references · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  4. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  5. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
  6. 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 37
  7. 2023Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animalscited by 10
  8. 2023Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotioncited by 10
  9. 2024Learning Highly Dynamic Behaviors for Quadrupedal Robotscited by 7
  10. 2025Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Controlcited by 3
  11. 2024Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformerscited by 1
  12. 2025In-between Motion Generation Based Multi-Style Quadruped Robot Locomotioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021A Pragmatic Look at Deep Imitation Learning
  2. 2025Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization
  3. 2022Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning
  4. 2026Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotion
  5. 2026Task-Centric Policy Optimization from Misaligned Motion Priors
  6. 2023DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies
  7. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitching
  8. 2025Pretraining in Actor-Critic Reinforcement Learning for Locomotion