# Permutohedral Attention Module for Efficient Non-Local Neural Networks

arXiv: https://arxiv.org/abs/1907.00641 · HTML: https://mlsanity.com/paper/1907.00641/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [Global Guidance Network for Breast Lesion Segmentation in Ultrasound Images](https://mlsanity.com/paper/2104.01896/) (2021)
- [Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation](https://mlsanity.com/paper/2107.00583/) (2021)
- [Towards Robotic Knee Arthroscopy: Multi-Scale Network for Tissue-Tool Segmentation](https://mlsanity.com/paper/2110.02657/) (2021)
- [Surface Vision Transformers: Flexible Attention-Based Modelling of Biomedical Surfaces](https://mlsanity.com/paper/2204.03408/) (2022)

## Abstract (arXiv)

Medical image processing tasks such as segmentation often require capturing non-local information. As organs, bones, and tissues share common characteristics such as intensity, shape, and texture, the contextual information plays a critical role in correctly labeling them. Segmentation and labeling is now typically done with convolutional neural networks (CNNs) but the context of the CNN is limited by the receptive field which itself is limited by memory requirements and other properties. In this paper, we propose a new attention module, that we call Permutohedral Attention Module (PAM), to efficiently capture non-local characteristics of the image. The proposed method is both memory and computationally efficient. We provide a GPU implementation of this module suitable for 3D medical imaging problems. We demonstrate the efficiency and scalability of our module with the challenging task of vertebrae segmentation and labeling where context plays a crucial role because of the very similar appearance of different vertebrae.