# Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers

arXiv: https://arxiv.org/abs/2210.13704 · HTML: https://mlsanity.com/paper/2210.13704/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [ShaRF: Shape-conditioned Radiance Fields from a Single View](https://mlsanity.com/paper/2102.08860/) (2021)
- [Learning Conditional Deformable Templates with Convolutional Networks](https://mlsanity.com/paper/1908.02738/) (2019)
- [DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images](https://mlsanity.com/paper/1810.00111/) (2018)
- [Bayesian Atlas Building with Hierarchical Priors for Subject-specific Regularization](https://mlsanity.com/paper/2107.05698/) (2021)
- [Fast Predictive Simple Geodesic Regression](https://mlsanity.com/paper/1711.05766/) (2017)

## Built on by (most cited first, 11 of 11)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction](https://mlsanity.com/paper/2309.03335/) (2023)
- [NeurEPDiff: Neural Operators to Predict Geodesics in Deformation Spaces](https://mlsanity.com/paper/2303.07115/) (2023)
- [RDA-INR: Riemannian Diffeomorphic Autoencoding via Implicit Neural Representations](https://mlsanity.com/paper/2305.12854/) (2023)
- [Invariant Shape Representation Learning For Image Classification](https://mlsanity.com/paper/2411.12201/) (2024)
- [MetaMorph: Learning Metamorphic Image Transformation With Appearance Changes](https://mlsanity.com/paper/2303.04849/) (2023)
- [MGAug: Multimodal Geometric Augmentation in Latent Spaces of Image Deformations](https://mlsanity.com/paper/2312.13440/) (2023)
- [SpaER: Learning Spatio-temporal Equivariant Representations for Fetal Brain Motion Tracking](https://mlsanity.com/paper/2407.20198/) (2024)
- [Learning Geodesics of Geometric Shape Deformations From Images](https://mlsanity.com/paper/2410.18797/) (2024)
- [CoRLD: Contrastive Representation Learning Of Deformable Shapes In Images](https://mlsanity.com/paper/2503.17162/) (2025)
- [Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification](https://mlsanity.com/paper/2607.07518/) (2026)

## Abstract (arXiv)

Deformable shapes provide important and complex geometric features of objects presented in images. However, such information is oftentimes missing or underutilized as implicit knowledge in many image analysis tasks. This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations; and (ii) gains increased model interpretability by allowing direct access to the underlying geometric features of image data. In particular, we develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations characterized by diffeomorphic transformations within each class. In contrast to previous approaches using pre-extracted shapes, our model provides a more fundamental approach by naturally learning the most relevant shape features jointly with an image classifier. We demonstrate the effectiveness of our method on both simulated 2D images and real 3D brain magnetic resonance (MR) images. Experimental results show that our model substantially improves the image classification accuracy with an additional benefit of increased model interpretability. Our code is publicly available at https://github.com/jw4hv/Geo-SIC