# STREAM: A Universal State-Space Model for Sparse Geometric Data

arXiv: https://arxiv.org/abs/2411.12603 · HTML: https://mlsanity.com/paper/2411.12603/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Vision Mamba backbones](https://mlsanity.com/ssm/vision-mamba-backbones/)

## Builds on (most cited first, 20 of 20)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [Learning to Detect Objects with a 1 Megapixel Event Camera](https://mlsanity.com/paper/2009.13436/) (2020)
- [PointMamba: A Simple State Space Model for Point Cloud Analysis](https://mlsanity.com/paper/2402.10739/) (2024)
- [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://mlsanity.com/paper/2212.08320/) (2022)
- [PointGPT: Auto-regressively Generative Pre-training from Point Clouds](https://mlsanity.com/paper/2305.11487/) (2023)
- [Point Cloud Mamba: Point Cloud Learning via State Space Model](https://mlsanity.com/paper/2403.00762/) (2024)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models](https://mlsanity.com/paper/2404.18508/) (2024)
- [Efficient recurrent architectures through activity sparsity and sparse back-propagation through time](https://mlsanity.com/paper/2206.06178/) (2022)
- [Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba](https://mlsanity.com/paper/2405.06116/) (2024)
- [S7: Selective and Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2410.03464/) (2024)
- [PLEIADES: Building Temporal Kernels with Orthogonal Polynomials](https://mlsanity.com/paper/2405.12179/) (2024)

## Built on by (most cited first, 2 of 2)

- [textttlrnnx: A library for Linear RNNs](https://mlsanity.com/paper/2602.08810/) (2026)
- [Point-Focused Attention Meets Context-Scan State Space: Robust Biological Visual Perception for Point Cloud Representation](https://mlsanity.com/paper/2610.11342/) (2026)

## Abstract (arXiv)

Handling sparse and unstructured geometric data, such as point clouds or event-based vision, is a pressing challenge in the field of machine vision. Recently, sequence models such as Transformers and state-space models entered the domain of geometric data. These methods require specialized preprocessing to create a sequential view of a set of points. Furthermore, prior works involving sequence models iterate geometric data with either uniform or learned step sizes, implicitly relying on the model to infer the underlying geometric structure. In this work, we propose to encode geometric structure explicitly into the parameterization of a state-space model. State-space models are based on linear dynamics governed by a one-dimensional variable such as time or a spatial coordinate. We exploit this dynamic variable to inject relative differences of coordinates into the step size of the state-space model. The resulting geometric operation computes interactions between all pairs of N points in O(N) steps. Our model deploys the Mamba selective state-space model with a modified CUDA kernel to efficiently map sparse geometric data to modern hardware. The resulting sequence model, which we call STREAM, achieves competitive results on a range of benchmarks from point-cloud classification to event-based vision and audio classification. STREAM demonstrates a powerful inductive bias for sparse geometric data by improving the PointMamba baseline when trained from scratch on the ModelNet40 and ScanObjectNN point cloud analysis datasets. It further achieves, for the first time, 100% test accuracy on all 11 classes of the DVS128 Gestures dataset.