# Imitation Game: A Model-based and Imitation Learning Deep Reinforcement Learning Hybrid

arXiv: https://arxiv.org/abs/2404.01794 · HTML: https://mlsanity.com/paper/2404.01794/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)
- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems](https://mlsanity.com/paper/2203.01387/) (2022)
- [Benchmarking Model-Based Reinforcement Learning](https://mlsanity.com/paper/1907.02057/) (2019)
- [Robust agents learn causal world models](https://mlsanity.com/paper/2402.10877/) (2024)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Stability Constrained Reinforcement Learning for Real-Time Voltage Control](https://mlsanity.com/paper/2109.14854/) (2021)
- [Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.10597/) (2019)
- [A Learning-based Power Management for Networked Microgrids Under Incomplete Information](https://mlsanity.com/paper/1810.01758/) (2018)
- [Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approach](https://mlsanity.com/paper/2006.07476/) (2020)
- [A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.12073/) (2021)
- [Adversarial Resilience Learning - Towards Systemic Vulnerability Analysis for Large and Complex Systems](https://mlsanity.com/paper/1811.06447/) (2018)
- [Deep Reinforcement Learning for Electric Transmission Voltage Control](https://mlsanity.com/paper/2006.06728/) (2020)
- [ANALYSE -- Learning to Attack Cyber-Physical Energy Systems With Intelligent Agents](https://mlsanity.com/paper/2305.09476/) (2023)

## Built on by (most cited first, 1 of 1)

- [On zero-shot learning in neural state estimation of power distribution systems](https://mlsanity.com/paper/2408.05787/) (2024)

## Abstract (arXiv)

Autonomous and learning systems based on Deep Reinforcement Learning have firmly established themselves as a foundation for approaches to creating resilient and efficient Cyber-Physical Energy Systems. However, most current approaches suffer from two distinct problems: Modern model-free algorithms such as Soft Actor Critic need a high number of samples to learn a meaningful policy, as well as a fallback to ward against concept drifts (e. g., catastrophic forgetting). In this paper, we present the work in progress towards a hybrid agent architecture that combines model-based Deep Reinforcement Learning with imitation learning to overcome both problems.