# Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired

arXiv: https://arxiv.org/abs/2103.04128 · HTML: https://mlsanity.com/paper/2103.04128/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Attention-guided Unified Network for Panoptic Segmentation](https://mlsanity.com/paper/1812.03904/) (2018)
- [BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation](https://mlsanity.com/paper/2003.14031/) (2020)
- [Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition](https://mlsanity.com/paper/2006.11538/) (2020)
- [Footprints and Free Space from a Single Color Image](https://mlsanity.com/paper/2004.06376/) (2020)
- [Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective](https://mlsanity.com/paper/2003.12122/) (2020)

## Built on by (most cited first, 5 of 5)

- [Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World](https://mlsanity.com/paper/2107.03172/) (2021)
- [Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance](https://mlsanity.com/paper/2108.09174/) (2021)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor](https://mlsanity.com/paper/2107.03180/) (2021)
- [MateRobot: Material Recognition in Wearable Robotics for People with Visual Impairments](https://mlsanity.com/paper/2302.14595/) (2023)

## Abstract (arXiv)

Classic computer vision algorithms, instance segmentation, and semantic segmentation can not provide a holistic understanding of the surroundings for the visually impaired. In this paper, we utilize panoptic segmentation to assist the navigation of visually impaired people by offering both things and stuff awareness in the proximity of the visually impaired efficiently. To this end, we propose an efficient Attention module -- Lintention which can model long-range interactions in linear time using linear space. Based on Lintention, we then devise a novel panoptic segmentation model which we term Panoptic Lintention Net. Experiments on the COCO dataset indicate that the Panoptic Lintention Net raises the Panoptic Quality (PQ) from 39.39 to 41.42 with 4.6% performance gain while only requiring 10% fewer GFLOPs and 25% fewer parameters in the semantic branch. Furthermore, a real-world test via our designed compact wearable panoptic segmentation system, indicates that our system based on the Panoptic Lintention Net accomplishes a relatively stable and exceptionally remarkable panoptic segmentation in real-world scenes.