# Rapid detection and recognition of whole brain activity in a freely behaving Caenorhabditis elegans

arXiv: https://arxiv.org/abs/2109.10474 · HTML: https://mlsanity.com/paper/2109.10474/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 22 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Rethinking Feature Discrimination and Polymerization for Large-scale Recognition](https://mlsanity.com/paper/1710.00870/) (2017)
- [Fast Online Deconvolution of Calcium Imaging Data](https://mlsanity.com/paper/1609.00639/) (2016)
- [Feature Incay for Representation Regularization](https://mlsanity.com/paper/1705.10284/) (2017)
- [Automatically tracking neurons in a moving and deforming brain](https://mlsanity.com/paper/1610.04579/) (2016)
- [Fast deep learning correspondence for neuron tracking and identification in C.elegans using synthetic training](https://mlsanity.com/paper/2101.08211/) (2021)

## Abstract (arXiv)

Advanced volumetric imaging methods and genetically encoded activity indicators have permitted a comprehensive characterization of whole brain activity at single neuron resolution in textitCaenorhabditis elegans. The constant motion and deformation of the nematode nervous system, however, impose a great challenge for consistent identification of densely packed neurons in a behaving animal. Here, we propose a cascade solution for long-term and rapid recognition of head ganglion neurons in a freely moving textitC. elegans. First, potential neuronal regions from a stack of fluorescence images are detected by a deep learning algorithm. Second, 2-dimensional neuronal regions are fused into 3-dimensional neuron entities. Third, by exploiting the neuronal density distribution surrounding a neuron and relative positional information between neurons, a multi-class artificial neural network transforms engineered neuronal feature vectors into digital neuronal identities. With a small number of training samples, our bottom-up approach is able to process each volume - 1024 × 1024 × 18 in voxels - in less than 1 second and achieves an accuracy of 91% in neuronal detection and above 80% in neuronal tracking over a long video recording. Our work represents a step towards rapid and fully automated algorithms for decoding whole brain activity underlying naturalistic behaviors.