# 3D View Prediction Models of the Dorsal Visual Stream

arXiv: https://arxiv.org/abs/2309.01782 · HTML: https://mlsanity.com/paper/2309.01782/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 7 of 7)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Learning Spatial Common Sense with Geometry-Aware Recurrent Networks](https://mlsanity.com/paper/1901.00003/) (2018)
- [Predictive coding: A Possible Explanation of Filling-in at the blind spot](https://mlsanity.com/paper/1507.01390/) (2015)

## Built on by (most cited first, 1 of 1)

- [Brain Decodes Deep Nets](https://mlsanity.com/paper/2312.01280/) (2023)

## Abstract (arXiv)

Deep neural network representations align well with brain activity in the ventral visual stream. However, the primate visual system has a distinct dorsal processing stream with different functional properties. To test if a model trained to perceive 3D scene geometry aligns better with neural responses in dorsal visual areas, we trained a self-supervised geometry-aware recurrent neural network (GRNN) to predict novel camera views using a 3D feature memory. We compared GRNN to self-supervised baseline models that have been shown to align well with ventral regions using the large-scale fMRI Natural Scenes Dataset (NSD). We found that while the baseline models accounted better for ventral brain regions, GRNN accounted for a greater proportion of variance in dorsal brain regions. Our findings demonstrate the potential for using task-relevant models to probe representational differences across visual streams.