# NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty

arXiv: https://arxiv.org/abs/2203.12117 · HTML: https://mlsanity.com/paper/2203.12117/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control](https://mlsanity.com/paper/1802.00265/) (2018)
- [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021)
- [Detecting and Adapting to Novelty in Games](https://mlsanity.com/paper/2106.02204/) (2021)
- [Representation Edit Distance as a Measure of Novelty](https://mlsanity.com/paper/2111.02770/) (2021)

## Built on by (most cited first, 14 of 14)

- [XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2312.12044/) (2023)
- [Neuro-Symbolic World Models for Adapting to Open World Novelty](https://mlsanity.com/paper/2301.06294/) (2023)
- [World Models: The Safety Perspective](https://mlsanity.com/paper/2411.07690/) (2024)
- [Novelty Detection in Reinforcement Learning with World Models](https://mlsanity.com/paper/2310.08731/) (2023)
- [NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds](https://mlsanity.com/paper/2401.03546/) (2024)
- [Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learning](https://mlsanity.com/paper/2404.02235/) (2024)
- [The Role of Exploration for Task Transfer in Reinforcement Learning](https://mlsanity.com/paper/2210.06168/) (2022)
- [NovPhy: A Testbed for Physical Reasoning in Open-world Environments](https://mlsanity.com/paper/2303.01711/) (2023)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [A Framework for Characterizing Novel Environment Transformations in General Environments](https://mlsanity.com/paper/2305.04315/) (2023)
- [Human in the Loop Novelty Generation](https://mlsanity.com/paper/2306.04813/) (2023)
- [Rapid Open-World Adaptation by Adaptation Principles Learning](https://mlsanity.com/paper/2312.11138/) (2023)
- [External Model Motivated Agents: Reinforcement Learning for Enhanced Environment Sampling](https://mlsanity.com/paper/2407.00264/) (2024)
- [Beyond Following: Mixing Active Initiative into Computational Creativity](https://mlsanity.com/paper/2409.16291/) (2024)

## Abstract (arXiv)

A robust body of reinforcement learning techniques have been developed to solve complex sequential decision making problems. However, these methods assume that train and evaluation tasks come from similarly or identically distributed environments. This assumption does not hold in real life where small novel changes to the environment can make a previously learned policy fail or introduce simpler solutions that might never be found. To that end we explore the concept of em novelty, defined in this work as the sudden change to the mechanics or properties of environment. We provide an ontology of for novelties most relevant to sequential decision making, which distinguishes between novelties that affect objects versus actions, unary properties versus non-unary relations, and the distribution of solutions to a task. We introduce NovGrid, a novelty generation framework built on MiniGrid, acting as a toolkit for rapidly developing and evaluating novelty-adaptation-enabled reinforcement learning techniques. Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates that can be applied to many MiniGrid-compliant environments. Finally, we present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques, and show characteristics of a baseline RL model using these metrics.