# Hierarchical Soft Actor-Critic: Adversarial Exploration via Mutual Information Optimization

arXiv: https://arxiv.org/abs/1906.07122 · HTML: https://mlsanity.com/paper/1906.07122/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2019.

## Built on by (most cited first, 2 of 2)

- [Developing cooperative policies for multi-stage reinforcement learning tasks](https://mlsanity.com/paper/2205.05230/) (2022)
- [SHIRO: Soft Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2212.12786/) (2022)

## Abstract (arXiv)

We describe a novel extension of soft actor-critics for hierarchical Deep Q-Networks (HDQN) architectures using mutual information metric. The proposed extension provides a suitable framework for encouraging explorations in such hierarchical networks. A natural utilization of this framework is an adversarial setting, where meta-controller and controller play minimax over the mutual information objective but cooperate on maximizing expected rewards.