# Seg-LSTM: Performance of xLSTM for Semantic Segmentation of Remotely Sensed Images

arXiv: https://arxiv.org/abs/2406.14086 · HTML: https://mlsanity.com/paper/2406.14086/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [xLSTM: Extended Long Short-Term Memory](https://mlsanity.com/paper/2405.04517/) (2024)
- [VM-UNet: Vision Mamba UNet for Medical Image Segmentation](https://mlsanity.com/paper/2402.02491/) (2024)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [Asymmetric Non-local Neural Networks for Semantic Segmentation](https://mlsanity.com/paper/1908.07678/) (2019)
- [MambaOut: Do We Really Need Mamba for Vision?](https://mlsanity.com/paper/2405.07992/) (2024)
- [Vision-LSTM: xLSTM as Generic Vision Backbone](https://mlsanity.com/paper/2406.04303/) (2024)
- [Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model](https://mlsanity.com/paper/2404.01705/) (2024)
- [Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2308.12113/) (2023)
- [Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study](https://mlsanity.com/paper/2405.08493/) (2024)
- [SBSS: Stacking-Based Semantic Segmentation Framework for Very High Resolution Remote Sensing Image](https://mlsanity.com/paper/2212.07623/) (2022)

## Built on by (most cited first, 5 of 5)

- [Mamba in Vision: A Comprehensive Survey of Techniques and Applications](https://mlsanity.com/paper/2410.03105/) (2024)
- [DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data](https://mlsanity.com/paper/2501.14256/) (2025)
- [Enhancing Environmental Monitoring through Multispectral Imaging: The WasteMS Dataset for Semantic Segmentation of Lakeside Waste](https://mlsanity.com/paper/2407.17028/) (2024)
- [MSCrackMamba: Leveraging Vision Mamba for Crack Detection in Fused Multispectral Imagery](https://mlsanity.com/paper/2412.06211/) (2024)
- [An overview of neural architectures for self-supervised audio representation learning from masked spectrograms](https://mlsanity.com/paper/2509.18691/) (2025)

## Abstract (arXiv)

Recent advancements in autoregressive networks with linear complexity have driven significant research progress, demonstrating exceptional performance in large language models. A representative model is the Extended Long Short-Term Memory (xLSTM), which incorporates gating mechanisms and memory structures, performing comparably to Transformer architectures in long-sequence language tasks. Autoregressive networks such as xLSTM can utilize image serialization to extend their application to visual tasks such as classification and segmentation. Although existing studies have demonstrated Vision-LSTM's impressive results in image classification, its performance in image semantic segmentation remains unverified. Our study represents the first attempt to evaluate the effectiveness of Vision-LSTM in the semantic segmentation of remotely sensed images. This evaluation is based on a specifically designed encoder-decoder architecture named Seg-LSTM, and comparisons with state-of-the-art segmentation networks. Our study found that Vision-LSTM's performance in semantic segmentation was limited and generally inferior to Vision-Transformers-based and Vision-Mamba-based models in most comparative tests. Future research directions for enhancing Vision-LSTM are recommended. The source code is available from https://github.com/zhuqinfeng1999/Seg-LSTM.