# Detecting and Adapting to Novelty in Games

arXiv: https://arxiv.org/abs/2106.02204 · HTML: https://mlsanity.com/paper/2106.02204/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [Learning to Simulate Dynamic Environments with GameGAN](https://mlsanity.com/paper/2005.12126/) (2020)
- [Graph Constrained Reinforcement Learning for Natural Language Action Spaces](https://mlsanity.com/paper/2001.08837/) (2020)
- [Relational Deep Reinforcement Learning](https://mlsanity.com/paper/1806.01830/) (2018)
- [Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1812.01628/) (2018)
- [Relational Forward Models for Multi-Agent Learning](https://mlsanity.com/paper/1809.11044/) (2018)
- [Learning to Predict Without Looking Ahead: World Models Without Forward Prediction](https://mlsanity.com/paper/1910.13038/) (2019)
- [Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess](https://mlsanity.com/paper/2009.04374/) (2020)

## Built on by (most cited first, 7 of 7)

- [NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty](https://mlsanity.com/paper/2203.12117/) (2022)
- [Neuro-Symbolic World Models for Adapting to Open World Novelty](https://mlsanity.com/paper/2301.06294/) (2023)
- [Measuring Diversity of Game Scenarios](https://mlsanity.com/paper/2404.15192/) (2024)
- [NovPhy: A Testbed for Physical Reasoning in Open-world Environments](https://mlsanity.com/paper/2303.01711/) (2023)
- [The Difficulty of Novelty Detection in Open-World Physical Domains: An Application to Angry Birds](https://mlsanity.com/paper/2106.08670/) (2021)
- [Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments](https://mlsanity.com/paper/2302.14208/) (2023)
- [Position: AI Is Not Ready for Strategic Conflicts](https://mlsanity.com/paper/2609.16189/) (2026)

## Abstract (arXiv)

Open-world novelty occurs when the rules of an environment can change abruptly, such as when a game player encounters "house rules". To address open-world novelty, game playing agents must be able to detect when novelty is injected, and to quickly adapt to the new rules. We propose a model-based reinforcement learning approach where game state and rules are represented as knowledge graphs. The knowledge graph representation of the state and rules allows novelty to be detected as changes in the knowledge graph, assists with the training of deep reinforcement learners, and enables imagination-based re-training where the agent uses the knowledge graph to perform look-ahead.