# From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

arXiv: https://arxiv.org/abs/1912.06207 · HTML: https://mlsanity.com/paper/1912.06207/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 11 of 11)

- [Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task](https://mlsanity.com/paper/2310.09336/) (2023)
- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [Spiking Vision Transformer with Saccadic Attention](https://mlsanity.com/paper/2502.12677/) (2025)
- [Brain-Model Evaluations Need the NeuroAI Turing Test](https://mlsanity.com/paper/2502.16238/) (2025)
- [Better artificial intelligence does not mean better models of biology](https://mlsanity.com/paper/2504.16940/) (2025)
- [Computing a human-like reaction time metric from stable recurrent vision models](https://mlsanity.com/paper/2306.11582/) (2023)
- [Signal Processing in the Retina: Interpretable Graph Classifier to Predict Ganglion Cell Responses](https://mlsanity.com/paper/2401.01813/) (2024)
- [Fine-Grained System Identification of Nonlinear Neural Circuits](https://mlsanity.com/paper/2106.05400/) (2021)
- [Emergent organization of receptive fields in networks of excitatory and inhibitory neurons](https://mlsanity.com/paper/2205.13614/) (2022)
- [Causal Interpretation of Neural Network Computations with Contribution Decomposition](https://mlsanity.com/paper/2603.06557/) (2026)
- [STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex](https://mlsanity.com/paper/2607.15631/) (2026)

## Abstract (arXiv)

Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.