# Visual novelty, curiosity, and intrinsic reward in machine learning and the brain

arXiv: https://arxiv.org/abs/1901.02478 · HTML: https://mlsanity.com/paper/1901.02478/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2019.

## Built on by (most cited first, 4 of 4)

- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks](https://mlsanity.com/paper/2407.21338/) (2024)
- [Dynamic Memory-based Curiosity: A Bootstrap Approach for Exploration](https://mlsanity.com/paper/2208.11349/) (2022)
- [Integrating Novelty and Surprise for Experience Prioritization and Exploration in Image-Based Reinforcement Learning](https://mlsanity.com/paper/2608.17373/) (2026)

## Abstract (arXiv)

A strong preference for novelty emerges in infancy and is prevalent across the animal kingdom. When incorporated into reinforcement-based machine learning algorithms, visual novelty can act as an intrinsic reward signal that vastly increases the efficiency of exploration and expedites learning, particularly in situations where external rewards are difficult to obtain. Here we review parallels between recent developments in novelty-driven machine learning algorithms and our understanding of how visual novelty is computed and signaled in the primate brain. We propose that in the visual system, novelty representations are not configured with the principal goal of detecting novel objects, but rather with the broader goal of flexibly generalizing novelty information across different states in the service of driving novelty-based learning.