# Cell Tracking according to Biological Needs -- Strong Mitosis-aware Multi-Hypothesis Tracker with Aleatoric Uncertainty

arXiv: https://arxiv.org/abs/2403.15011 · HTML: https://mlsanity.com/paper/2403.15011/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 69 references that are in the map.
- First posted on arXiv 22 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Rethinking Bayesian Deep Learning Methods for Semi-Supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2206.09293/) (2022)
- [Making Higher Order MOT Scalable: An Efficient Approximate Solver for Lifted Disjoint Paths](https://mlsanity.com/paper/2108.10606/) (2021)
- [Graph Neural Network for Cell Tracking in Microscopy Videos](https://mlsanity.com/paper/2202.04731/) (2022)
- [Microscopy Cell Segmentation via Convolutional LSTM Networks](https://mlsanity.com/paper/1805.11247/) (2018)
- [EmbedTrack -- Simultaneous Cell Segmentation and Tracking Through Learning Offsets and Clustering Bandwidths](https://mlsanity.com/paper/2204.10713/) (2022)
- [TrackFlow: Multi-Object Tracking with Normalizing Flows](https://mlsanity.com/paper/2308.11513/) (2023)
- [Poisson multi-Bernoulli mixture trackers: continuity through random finite sets of trajectories](https://mlsanity.com/paper/1812.05131/) (2018)
- [CellTrack R-CNN: A Novel End-To-End Deep Neural Network for Cell Segmentation and Tracking in Microscopy Images](https://mlsanity.com/paper/2102.10377/) (2021)
- [J Regularization Improves Imbalanced Multiclass Segmentation](https://mlsanity.com/paper/1910.09783/) (2019)
- [Joint Multi-frame Detection and Segmentation for Multi-cell Tracking](https://mlsanity.com/paper/1906.10886/) (2019)
- [Tracking Cells and their Lineages via Labeled Random Finite Sets](https://mlsanity.com/paper/2104.10964/) (2021)
- [Compensation Learning in Semantic Segmentation](https://mlsanity.com/paper/2304.13428/) (2023)
- [Fuzzy-based Propagation of Prior Knowledge to Improve Large-Scale Image Analysis Pipelines](https://mlsanity.com/paper/1608.01276/) (2016)
- [Tracking-Assisted Segmentation of Biological Cells](https://mlsanity.com/paper/1910.08735/) (2019)
- [Multi-Target Tracking with Time-Varying Clutter Rate and Detection Profile: Application to Time-lapse Cell Microscopy Sequences](https://mlsanity.com/paper/1507.06397/) (2015)

## Built on by (most cited first, 3 of 3)

- [UncertainSAM: Fast and Efficient Uncertainty Quantification of the Segment Anything Model](https://mlsanity.com/paper/2505.05049/) (2025)
- [How To Make Your Cell Tracker Say "I dunno!"](https://mlsanity.com/paper/2503.09244/) (2025)
- [Higher-Order Cell Tracking Transformer](https://mlsanity.com/paper/2607.11754/) (2026)

## Abstract (arXiv)

Cell tracking and segmentation assist biologists in extracting insights from large-scale microscopy time-lapse data. Driven by local accuracy metrics, current tracking approaches often suffer from a lack of long-term consistency and the ability to reconstruct lineage trees correctly. To address this issue, we introduce an uncertainty estimation technique for motion estimation frameworks and extend the multi-hypothesis tracking framework. Our uncertainty estimation lifts motion representations into probabilistic spatial densities using problem-specific test-time augmentations. Moreover, we introduce a novel mitosis-aware assignment problem formulation that allows multi-hypothesis trackers to model cell splits and to resolve false associations and mitosis detections based on long-term conflicts. In our framework, explicit biological knowledge is modeled in assignment costs. We evaluate our approach on nine competitive datasets and demonstrate that we outperform the current state-of-the-art on biologically inspired metrics substantially, achieving improvements by a factor of approximately 6 and uncover new insights into the behavior of motion estimation uncertainty.