Discriminator Soft Actor Critic without Extrinsic Rewards
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement learning and generative adversarial imitation learning (GAIL), can learn from only a few expert data. However, they often need to interact with the environment. Soft Q imitation learning addressed the problems, and it was shown that it could learn efficiently by combining Behavioral Cloning and soft Q-learning with constant rewards. In order to make this algorithm more robust to distribution shift, we propose Discriminator Soft Actor Critic (DSAC). It uses a reward function based on adversarial inverse reinforcement learning instead of constant rewards. We evaluated it on PyBullet environments with only four expert trajectories.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 19 Jan 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 1; 2021: 0; 2022: 2.
3 papers.
- 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4
- 2022Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learningcited by 2
- 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policycited by 1
- 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policy21 Jun 2022
- 2022Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning22 Mar 2022
- 2020Forward and inverse reinforcement learning sharing network weights and hyperparameters17 Aug 2020
Similar, not cited
Close in wording, no citation either way
- 2024Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator
- 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
- 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
- 2023Coherent Soft Imitation Learning
- 2021IQ-Learn: Inverse soft-Q Learning for Imitation
- 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
- 2026Scalable Causal Imitation Learning
- 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards