mlsanityData to 13 Oct 2026

Memory Encoding Model

arXiv 2308.01175 ↗First posted 2 Aug 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We explore a new class of brain encoding model by adding memory-related information as input. Memory is an essential brain mechanism that works alongside visual stimuli. During a vision-memory cognitive task, we found the non-visual brain is largely predictable using previously seen images. Our Memory Encoding Model (Mem) won the Algonauts 2023 visual brain competition even without model ensemble (single model score 66.8, ensemble score 70.8). Our ensemble model without memory input (61.4) can also stand a 3rd place. Furthermore, we observe periodic delayed brain response correlated to 6th-7th prior image, and hippocampus also showed correlated activity timed with this periodicity. We conjuncture that the periodic replay could be related to memory mechanism to enhance the working memory.

Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 11 of its 33 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

11 of 33 references.

References · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  3. 2022Scalable Diffusion Models with Transformerscited by 5,973
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,384
  5. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 437
  6. 2018BOLD5000: A public fMRI dataset of 5000 imagescited by 63
  7. 2016Neural Encoding and Decoding with Deep Learning for Dynamic Natural Visioncited by 58
  8. 2023The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenescited by 34
  9. 2021The Algonauts Project 2021 Challenge: How the Human Brain Makes Sense of a World in Motioncited by 17
  10. 2022The Sensorium competition on predicting large-scale mouse primary visual cortex activitycited by 13
  11. 2023Retinotopy Inspired Brain Encoding Model and the All-for-One Training Recipecited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 3; 2026: 1.

7 papers.

  1. 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 105
  2. 2025MindSimulator: Exploring Brain Concept Localization via Synthetic FMRIcited by 15
  3. 2026A foundation model of vision, audition, and language for in-silico neurosciencecited by 12
  4. 2024Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversioncited by 9
  5. 2025SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learningcited by 9
  6. 2025Predicting Brain Responses To Natural Movies With Multimodal LLMscited by 5
  7. 2023A Parameter-efficient Multi-subject Model for Predicting fMRI Activitycited by 2