# Learning Scene Gist with Convolutional Neural Networks to Improve Object Recognition

arXiv: https://arxiv.org/abs/1803.01967 · HTML: https://mlsanity.com/paper/1803.01967/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2018.

## Built on by (most cited first, 5 of 5)

- [When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes](https://mlsanity.com/paper/2104.02215/) (2021)
- [A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains](https://mlsanity.com/paper/2310.13849/) (2023)
- [Identifying and Benchmarking Natural Out-of-Context Prediction Problems](https://mlsanity.com/paper/2110.13223/) (2021)
- [Context-driven Visual Object Recognition based on Knowledge Graphs](https://mlsanity.com/paper/2210.11233/) (2022)
- [A Comprehensive Sustainable Framework for Machine Learning and Artificial Intelligence](https://mlsanity.com/paper/2407.12445/) (2024)

## Abstract (arXiv)

Advancements in convolutional neural networks (CNNs) have made significant strides toward achieving high performance levels on multiple object recognition tasks. While some approaches utilize information from the entire scene to propose regions of interest, the task of interpreting a particular region or object is still performed independently of other objects and features in the image. Here we demonstrate that a scene's 'gist' can significantly contribute to how well humans can recognize objects. These findings are consistent with the notion that humans foveate on an object and incorporate information from the periphery to aid in recognition. We use a biologically inspired two-part convolutional neural network ('GistNet') that models the fovea and periphery to provide a proof-of-principle demonstration that computational object recognition can significantly benefit from the gist of the scene as contextual information. Our model yields accuracy improvements of up to 50% in certain object categories when incorporating contextual gist, while only increasing the original model size by 5%. This proposed model mirrors our intuition about how the human visual system recognizes objects, suggesting specific biologically plausible constraints to improve machine vision and building initial steps towards the challenge of scene understanding.