mlsanityData to 7 Oct 2026

V1T: large-scale mouse V1 response prediction using a Vision Transformer

arXiv 2302.03023 ↗First posted 6 Feb 2023cs.CV cs.LG cs.NEcited by 5

Not on the map. Why?

Abstract (arXiv)

Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previous convolution-based models by more than 12.7% in prediction performance. Moreover, we show that the self-attention weights learned by the Transformer correlate with the population receptive fields. Our model thus sets a new benchmark for neural response prediction and can be used jointly with behavioral and neural recordings to reveal meaningful characteristic features of the visual cortex.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 30 of its 85 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

30 of 85 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016Layer Normalizationcited by 5,579
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  7. 2016Deep Networks with Stochastic Depthcited by 973
  8. 2020On Layer Normalization in the Transformer Architecturecited by 906
  9. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  10. 2020Quantifying Attention Flow in Transformerscited by 759
  11. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  12. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
18 more
  1. 2020A Survey on Visual Transformercited by 653
  2. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  3. 2018Markerless tracking of user-defined features with deep learningcited by 203
  4. 2021Vision Transformer for Small-Size Datasetscited by 122
  5. 2022Learnable latent embeddings for joint behavioral and neural analysiscited by 113
  6. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  7. 2022Cramming: Training a Language Model on a Single GPU in One Daycited by 81
  8. 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
  9. 2014Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognitioncited by 50
  10. 2021Representation learning for neural population activity with Neural Data Transformerscited by 48
  11. 2019Learning From Brains How to Regularize Machinescited by 37
  12. 2017Neural system identification for large populations separating "what" and "where"cited by 28
  13. 2021Towards robust vision by multi-task learning on monkey visual cortexcited by 28
  14. 2022STNDT: Modeling Neural Population Activity with a Spatiotemporal Transformercited by 23
  15. 2018A rotation-equivariant convolutional neural network model of primary visual cortexcited by 15
  16. 2022The Sensorium competition on predicting large-scale mouse primary visual cortex activitycited by 13
  17. 2022Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4cited by 7
  18. 2021Neuronal Learning Analysis using Cycle-Consistent Adversarial Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Self-Attention-Based Contextual Modulation Improves Neural System Identificationcited by 0
  2. 2024Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factorscited by 0
  3. 2024QuantFormer: Learning to Quantize for Neural Activity Forecasting in Mouse Visual Cortexcited by 0
  4. 2025MEIcoder: Decoding Visual Stimuli from Neural Activity by Leveraging Most Exciting Inputscited by 0
  5. 2025Manifolds and Modules: How Function Develops in a Neural Foundation Modelcited by 0