# Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation

arXiv: https://arxiv.org/abs/1506.07452 · HTML: https://mlsanity.com/paper/1506.07452/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2015.

## Built on by (most cited first, 16 of 16)

- [IterMask2: Iterative Unsupervised Anomaly Segmentation via Spatial and Frequency Masking for Brain Lesions in MRI](https://mlsanity.com/paper/2406.02422/) (2024)
- [Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling](https://mlsanity.com/paper/2103.08877/) (2021)
- [LiteLSTM Architecture for Deep Recurrent Neural Networks](https://mlsanity.com/paper/2201.11624/) (2022)
- [ME-Net: Multi-Encoder Net Framework for Brain Tumor Segmentation](https://mlsanity.com/paper/2203.11213/) (2022)
- [Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B](https://mlsanity.com/paper/2504.04918/) (2025)
- [Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation](https://mlsanity.com/paper/2201.00317/) (2022)
- [pLSTM: parallelizable Linear Source Transition Mark networks](https://mlsanity.com/paper/2506.11997/) (2025)
- [Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving](https://mlsanity.com/paper/2101.06865/) (2021)
- [LiteLSTM Architecture Based on Weights Sharing for Recurrent Neural Networks](https://mlsanity.com/paper/2301.04794/) (2023)
- [Recurrent U-net for automatic pelvic floor muscle segmentation on 3D ultrasound](https://mlsanity.com/paper/2107.13833/) (2021)
- [An overview of deep learning in medical imaging](https://mlsanity.com/paper/2202.08546/) (2022)
- [DartsReNet: Exploring new RNN cells in ReNet architectures](https://mlsanity.com/paper/2304.05838/) (2023)
- [Autonomous Robotic Drilling System for Mice Cranial Window Creation](https://mlsanity.com/paper/2406.14135/) (2024)
- [Full-Head Segmentation of MRI with Abnormal Brain Anatomy: Model and Data Release](https://mlsanity.com/paper/2501.18716/) (2025)
- [Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion](https://mlsanity.com/paper/2504.08964/) (2025)
- [Hybrid Context-Fusion Attention (CFA) U-Net and Clustering for Robust Seismic Horizon Interpretation](https://mlsanity.com/paper/2512.00191/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D videos to segment them. They have a fixed input size and typically perceive only small local contexts of the pixels to be classified as foreground or background. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive the entire spatio-temporal context of each pixel in a few sweeps through all pixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite these theoretical advantages, however, unlike CNNs, previous MD-LSTM variants were hard to parallelize on GPUs. Here we re-arrange the traditional cuboid order of computations in MD-LSTM in pyramidal fashion. The resulting PyraMiD-LSTM is easy to parallelize, especially for 3D data such as stacks of brain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).