# Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

arXiv: https://arxiv.org/abs/1806.02070 · HTML: https://mlsanity.com/paper/1806.02070/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2018.

## Built on by (most cited first, 14 of 14)

- [Region Proposal Rectification Towards Robust Instance Segmentation of Biological Images](https://mlsanity.com/paper/2203.02846/) (2022)
- [CellTrack R-CNN: A Novel End-To-End Deep Neural Network for Cell Segmentation and Tracking in Microscopy Images](https://mlsanity.com/paper/2102.10377/) (2021)
- [Large-Scale Multi-Hypotheses Cell Tracking Using Ultrametric Contours Maps](https://mlsanity.com/paper/2308.04526/) (2023)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations](https://mlsanity.com/paper/2101.00567/) (2021)
- [Object-Guided Instance Segmentation With Auxiliary Feature Refinement for Biological Images](https://mlsanity.com/paper/2106.07159/) (2021)
- [Search for temporal cell segmentation robustness in phase-contrast microscopy videos](https://mlsanity.com/paper/2112.08817/) (2021)
- [H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation](https://mlsanity.com/paper/2206.01309/) (2022)
- [Stochastic Neural Networks for Automatic Cell Tracking in Microscopy Image Sequences of Bacterial Colonies](https://mlsanity.com/paper/2104.13482/) (2021)
- [Cell Tracking-by-detection using Elliptical Bounding Boxes](https://mlsanity.com/paper/2310.04895/) (2023)
- [Graph Relation Distillation for Efficient Biomedical Instance Segmentation](https://mlsanity.com/paper/2401.06370/) (2024)
- [Instance segmentation with the number of clusters incorporated in embedding learning](https://mlsanity.com/paper/2103.14869/) (2021)
- [Look in Different Views: Multi-Scheme Regression Guided Cell Instance Segmentation](https://mlsanity.com/paper/2208.08078/) (2022)
- [Cell as Point: One-Stage Framework for Efficient Cell Tracking](https://mlsanity.com/paper/2411.14833/) (2024)

## Abstract (arXiv)

Different to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same class. In this work, we propose a novel recurrent fully convolutional network architecture for tracking such instance segmentations over time. The network architecture incorporates convolutional gated recurrent units (ConvGRU) into a stacked hourglass network to utilize temporal video information. Furthermore, we train the network with a novel embedding loss based on cosine similarities, such that the network predicts unique embeddings for every instance throughout videos. Afterwards, these embeddings are clustered among subsequent video frames to create the final tracked instance segmentations. We evaluate the recurrent hourglass network by segmenting left ventricles in MR videos of the heart, where it outperforms a network that does not incorporate video information. Furthermore, we show applicability of the cosine embedding loss for segmenting leaf instances on still images of plants. Finally, we evaluate the framework for instance segmentation and tracking on six datasets of the ISBI celltracking challenge, where it shows state-of-the-art performance.