# Loc-VAE: Learning Structurally Localized Representation from 3D Brain MR Images for Content-Based Image Retrieval

arXiv: https://arxiv.org/abs/2210.00506 · HTML: https://mlsanity.com/paper/2210.00506/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 2 Oct 2022.

## Builds on (most cited first, 6 of 6)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification](https://mlsanity.com/paper/1701.06643/) (2017)
- [End-To-End Alzheimer's Disease Diagnosis and Biomarker Identification](https://mlsanity.com/paper/1810.00523/) (2018)
- [Efficient feature embedding of 3D brain MRI images for content-based image retrieval with deep metric learning](https://mlsanity.com/paper/1912.01824/) (2019)

## Built on by (most cited first, 2 of 2)

- [iCBIR-Sli: Interpretable Content-Based Image Retrieval with 2D Slice Embeddings](https://mlsanity.com/paper/2501.01642/) (2025)
- [NeuroCBIR: A Fast and Accurate Image Retrieval System for Whole-Brain and Region-Specific MRI](https://mlsanity.com/paper/2610.06502/) (2026)

## Abstract (arXiv)

Content-based image retrieval (CBIR) systems are an emerging technology that supports reading and interpreting medical images. Since 3D brain MR images are high dimensional, dimensionality reduction is necessary for CBIR using machine learning techniques. In addition, for a reliable CBIR system, each dimension in the resulting low-dimensional representation must be associated with a neurologically interpretable region. We propose a localized variational autoencoder (Loc-VAE) that provides neuroanatomically interpretable low-dimensional representation from 3D brain MR images for clinical CBIR. Loc-VAE is based on β-VAE with the additional constraint that each dimension of the low-dimensional representation corresponds to a local region of the brain. The proposed Loc-VAE is capable of acquiring representation that preserves disease features and is highly localized, even under high-dimensional compression ratios (4096:1). The low-dimensional representation obtained by Loc-VAE improved the locality measure of each dimension by 4.61 points compared to naive β-VAE, while maintaining comparable brain reconstruction capability and information about the diagnosis of Alzheimer's disease.