# Goal-Conditioned Reinforcement Learning in the Presence of an Adversary

arXiv: https://arxiv.org/abs/2211.06929 · HTML: https://mlsanity.com/paper/2211.06929/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 13 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Language as an Abstraction for Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07343/) (2019)
- [Learning to Execute](https://mlsanity.com/paper/1410.4615/) (2014)
- [Hierarchical and Safe Motion Control for Cooperative Locomotion of Robotic Guide Dogs and Humans: A Hybrid Systems Approach](https://mlsanity.com/paper/1904.03158/) (2019)
- [Deploying the NASA Valkyrie Humanoid for IED Response: An Initial Approach and Evaluation Summary](https://mlsanity.com/paper/1910.00761/) (2019)
- [Cooperative Pollution Source Localization and Cleanup with a Bio-inspired Swarm Robot Aggregation](https://mlsanity.com/paper/1907.09585/) (2019)
- [Deep Reinforcement Learning Based Robot Arm Manipulation with Efficient Training Data through Simulation](https://mlsanity.com/paper/1907.06884/) (2019)

## Abstract (arXiv)

Reinforcement learning has seen increasing applications in real-world contexts over the past few years. However, physical environments are often imperfect and policies that perform well in simulation might not achieve the same performance when applied elsewhere. A common approach to combat this is to train agents in the presence of an adversary. An adversary acts to destabilise the agent, which learns a more robust policy and can better handle realistic conditions. Many real-world applications of reinforcement learning also make use of goal-conditioning: this is particularly useful in the context of robotics, as it allows the agent to act differently, depending on which goal is selected. Here, we focus on the problem of goal-conditioned learning in the presence of an adversary. We first present DigitFlip and CLEVR-Play, two novel goal-conditioned environments that support acting against an adversary. Next, we propose EHER and CHER -- two HER-based algorithms for goal-conditioned learning -- and evaluate their performance. Finally, we unify the two threads and introduce IGOAL: a novel framework for goal-conditioned learning in the presence of an adversary. Experimental results show that combining IGOAL with EHER allows agents to significantly outperform existing approaches, when acting against both random and competent adversaries.