# Graph Relation Distillation for Efficient Biomedical Instance Segmentation

arXiv: https://arxiv.org/abs/2401.06370 · HTML: https://mlsanity.com/paper/2401.06370/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 27 of 27)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Similarity-Preserving Knowledge Distillation](https://mlsanity.com/paper/1907.09682/) (2019)
- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [Semantic Instance Segmentation with a Discriminative Loss Function](https://mlsanity.com/paper/1708.02551/) (2017)
- [Cross-Image Relational Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2204.06986/) (2022)
- [Masked Generative Distillation](https://mlsanity.com/paper/2205.01529/) (2022)
- [DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation](https://mlsanity.com/paper/1604.02677/) (2016)
- [CrossKD: Cross-Head Knowledge Distillation for Object Detection](https://mlsanity.com/paper/2306.11369/) (2023)
- [Efficient Medical Image Segmentation Based on Knowledge Distillation](https://mlsanity.com/paper/2108.09987/) (2021)
- [Paraphrase Generation with Latent Bag of Words](https://mlsanity.com/paper/2001.01941/) (2020)
- [Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks](https://mlsanity.com/paper/1806.02070/) (2018)
- [Instance Segmentation of Biological Images Using Harmonic Embeddings](https://mlsanity.com/paper/1904.05257/) (2019)
- [Instance Segmentation of Biomedical Images with an Object-aware Embedding Learned with Local Constraints](https://mlsanity.com/paper/2004.09821/) (2020)
- [Rethinking Ensemble-Distillation for Semantic Segmentation Based Unsupervised Domain Adaptation](https://mlsanity.com/paper/2104.14203/) (2021)
- [Sparse Object-level Supervision for Instance Segmentation with Pixel Embeddings](https://mlsanity.com/paper/2103.14572/) (2021)
- [Deep Structured Instance Graph for Distilling Object Detectors](https://mlsanity.com/paper/2109.12862/) (2021)
- [Learning and Segmenting Dense Voxel Embeddings for 3D Neuron Reconstruction](https://mlsanity.com/paper/1909.09872/) (2019)
- [Advanced Deep Networks for 3D Mitochondria Instance Segmentation](https://mlsanity.com/paper/2104.07961/) (2021)

## Built on by (most cited first, 1 of 1)

- [SomaNet: Weakly Supervised Learning for Instance Soma Segmentation in 3D Electron Microscopy with Partial Annotations](https://mlsanity.com/paper/2609.23019/) (2026)

## Abstract (arXiv)

Instance-aware embeddings predicted by deep neural networks have revolutionized biomedical instance segmentation, but its resource requirements are substantial. Knowledge distillation offers a solution by transferring distilled knowledge from heavy teacher networks to lightweight yet high-performance student networks. However, existing knowledge distillation methods struggle to extract knowledge for distinguishing instances and overlook global relation information. To address these challenges, we propose a graph relation distillation approach for efficient biomedical instance segmentation, which considers three essential types of knowledge: instance-level features, instance relations, and pixel-level boundaries. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level: instance graph distillation (IGD) and affinity graph distillation (AGD). IGD constructs a graph representing instance features and relations, transferring these two types of knowledge by enforcing instance graph consistency. AGD constructs an affinity graph representing pixel relations to capture structured knowledge of instance boundaries, transferring boundary-related knowledge by ensuring pixel affinity consistency. Experimental results on a number of biomedical datasets validate the effectiveness of our approach, enabling student models with less than  1% parameters and less than 10% inference time while achieving promising performance compared to teacher models.