# Generic decoding of seen and imagined objects using hierarchical visual features

arXiv: https://arxiv.org/abs/1510.06479 · HTML: https://mlsanity.com/paper/1510.06479/

## Key facts

- Cited by 120 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2015.

## Built on by (most cited first, 30 of 120)

- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Natural scene reconstruction from fMRI signals using generative latent diffusion](https://mlsanity.com/paper/2303.05334/) (2023)
- [Mind Reader: Reconstructing complex images from brain activities](https://mlsanity.com/paper/2210.01769/) (2022)
- [Decoding Natural Images from EEG for Object Recognition](https://mlsanity.com/paper/2308.13234/) (2023)
- [Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity](https://mlsanity.com/paper/2305.11675/) (2023)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [MindBridge: A Cross-Subject Brain Decoding Framework](https://mlsanity.com/paper/2404.07850/) (2024)
- [Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features](https://mlsanity.com/paper/2210.06756/) (2022)
- [Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANs](https://mlsanity.com/paper/2202.12692/) (2022)
- [MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusion](https://mlsanity.com/paper/2308.04249/) (2023)
- [DREAM: Visual Decoding from Reversing Human Visual System](https://mlsanity.com/paper/2310.02265/) (2023)
- [UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity](https://mlsanity.com/paper/2308.07428/) (2023)
- [BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding](https://mlsanity.com/paper/2302.12971/) (2023)
- [Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities](https://mlsanity.com/paper/2305.17214/) (2023)
- [Brain Captioning: Decoding human brain activity into images and text](https://mlsanity.com/paper/2305.11560/) (2023)
- [MindTuner: Cross-Subject Visual Decoding with Visual Fingerprint and Semantic Correction](https://mlsanity.com/paper/2404.12630/) (2024)
- [Controllable Mind Visual Diffusion Model](https://mlsanity.com/paper/2305.10135/) (2023)
- [A Systematic Collection of Medical Image Datasets for Deep Learning](https://mlsanity.com/paper/2106.12864/) (2021)
- [NeuroClips: Towards High-fidelity and Smooth fMRI-to-Video Reconstruction](https://mlsanity.com/paper/2410.19452/) (2024)
- [Improving visual image reconstruction from human brain activity using latent diffusion models via multiple decoded inputs](https://mlsanity.com/paper/2306.11536/) (2023)
- [Neuro-3D: Towards 3D Visual Decoding from EEG Signals](https://mlsanity.com/paper/2411.12248/) (2024)
- [Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activity](https://mlsanity.com/paper/2403.20022/) (2024)
- [Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)](https://mlsanity.com/paper/2307.10246/) (2023)
- [Through their eyes: multi-subject Brain Decoding with simple alignment techniques](https://mlsanity.com/paper/2309.00627/) (2023)
- [Visual Neural Decoding via Improved Visual-EEG Semantic Consistency](https://mlsanity.com/paper/2408.06788/) (2024)
- [fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding](https://mlsanity.com/paper/2311.00342/) (2023)
- [Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy](https://mlsanity.com/paper/2401.00430/) (2023)
- [The Wisdom of a Crowd of Brains: A Universal Brain Encoder](https://mlsanity.com/paper/2406.12179/) (2024)

## Abstract (arXiv)

Object recognition is a key function in both human and machine vision. While recent studies have achieved fMRI decoding of seen and imagined contents, the prediction is limited to training examples. We present a decoding approach for arbitrary objects, using the machine vision principle that an object category is represented by a set of features rendered invariant through hierarchical processing. We show that visual features including those from a convolutional neural network can be predicted from fMRI patterns and that greater accuracy is achieved for low/high-level features with lower/higher-level visual areas, respectively. Predicted features are used to identify seen/imagined object categories (extending beyond decoder training) from a set of computed features for numerous object images. Furthermore, the decoding of imagined objects reveals progressive recruitment of higher to lower visual representations. Our results demonstrate a homology between human and machine vision and its utility for brain-based information retrieval.