# Invariant Shape Representation Learning For Image Classification

arXiv: https://arxiv.org/abs/2411.12201 · HTML: https://mlsanity.com/paper/2411.12201/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 74 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [Interventional Few-Shot Learning](https://mlsanity.com/paper/2009.13000/) (2020)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [Detect and Correct Bias in Multi-Site Neuroimaging Datasets](https://mlsanity.com/paper/2002.05049/) (2020)
- [Out-of-distribution Generalization with Causal Invariant Transformations](https://mlsanity.com/paper/2203.11528/) (2022)
- [Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching](https://mlsanity.com/paper/2106.08600/) (2021)
- [Confidence Calibration for Domain Generalization under Covariate Shift](https://mlsanity.com/paper/2104.00742/) (2021)
- [Semi-supervised Task-driven Data Augmentation for Medical Image Segmentation](https://mlsanity.com/paper/2007.05363/) (2020)
- [Enhancing MR Image Segmentation with Realistic Adversarial Data Augmentation](https://mlsanity.com/paper/2108.03429/) (2021)
- [SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction](https://mlsanity.com/paper/2309.03335/) (2023)

## Built on by (most cited first, 5 of 5)

- [ShapeEmbed: a self-supervised learning framework for 2D contour quantification](https://mlsanity.com/paper/2507.01009/) (2025)
- [CoRLD: Contrastive Representation Learning Of Deformable Shapes In Images](https://mlsanity.com/paper/2503.17162/) (2025)
- [Learning to Unify Deformable Shape and Texture Representations for Cardiac Video Classification](https://mlsanity.com/paper/2607.07518/) (2026)
- [Learning Group Actions In Disentangled Latent Image Representations](https://mlsanity.com/paper/2512.04015/) (2025)
- [Learning To Focus: Anatomy-Guided Attention Regularization for Medical Image Classification](https://mlsanity.com/paper/2607.10851/) (2026)

## Abstract (arXiv)

Geometric shape features have been widely used as strong predictors for image classification. Nevertheless, most existing classifiers such as deep neural networks (DNNs) directly leverage the statistical correlations between these shape features and target variables. However, these correlations can often be spurious and unstable across different environments (e.g., in different age groups, certain types of brain changes have unstable relations with neurodegenerative disease); hence leading to biased or inaccurate predictions. In this paper, we introduce a novel framework that for the first time develops invariant shape representation learning (ISRL) to further strengthen the robustness of image classifiers. In contrast to existing approaches that mainly derive features in the image space, our model ISRL is designed to jointly capture invariant features in latent shape spaces parameterized by deformable transformations. To achieve this goal, we develop a new learning paradigm based on invariant risk minimization (IRM) to learn invariant representations of image and shape features across multiple training distributions/environments. By embedding the features that are invariant with regard to target variables in different environments, our model consistently offers more accurate predictions. We validate our method by performing classification tasks on both simulated 2D images, real 3D brain and cine cardiovascular magnetic resonance images (MRIs). Our code is publicly available at https://github.com/tonmoy-hossain/ISRL.