# Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection

arXiv: https://arxiv.org/abs/2406.10700 · HTML: https://mlsanity.com/paper/2406.10700/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 66 of its 81 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/)

## Builds on (most cited first, 30 of 66)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Hungry Hungry Hippos: Towards Language Modeling with State Space Models](https://mlsanity.com/paper/2212.14052/) (2022)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection](https://mlsanity.com/paper/2102.00463/) (2021)
- [VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.11301/) (2023)

## Built on by (most cited first, 30 of 66)

- [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2407.18232/) (2024)
- [Mamba in Vision: A Comprehensive Survey of Techniques and Applications](https://mlsanity.com/paper/2410.03105/) (2024)
- [Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model](https://mlsanity.com/paper/2503.16282/) (2025)
- [Fast Multi-view Consistent 3D Editing with Video Priors](https://mlsanity.com/paper/2511.23172/) (2025)
- [UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2503.12009/) (2025)
- [General Geometry-aware Weakly Supervised 3D Object Detection](https://mlsanity.com/paper/2407.13748/) (2024)
- [MambaPlace:Text-to-Point-Cloud Cross-Modal Place Recognition with Attention Mamba Mechanisms](https://mlsanity.com/paper/2408.15740/) (2024)
- [Omni-3DEdit: Generalized Versatile 3D Editing in One-Pass](https://mlsanity.com/paper/2603.17841/) (2026)
- [MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reordering](https://mlsanity.com/paper/2408.11464/) (2024)
- [Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation](https://mlsanity.com/paper/2409.11018/) (2024)
- [Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching](https://mlsanity.com/paper/2410.06285/) (2024)
- [MambaBEV: An EV-based 3D detection model with Mamba2](https://mlsanity.com/paper/2410.12673/) (2024)
- [Efficient Spiking Point Mamba for Point Cloud Analysis](https://mlsanity.com/paper/2504.14371/) (2025)
- [UD-Mamba: A pixel-level uncertainty-driven Mamba model for medical image segmentation](https://mlsanity.com/paper/2502.02024/) (2025)
- [NoKSR: Kernel-Free Neural Surface Reconstruction via Point Cloud Serialization](https://mlsanity.com/paper/2502.12534/) (2025)
- [GAMBAS: Generalised-Hilbert Mamba for Super-resolution of Paediatric Ultra-Low-Field MRI](https://mlsanity.com/paper/2504.04523/) (2025)
- [MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection](https://mlsanity.com/paper/2507.04369/) (2025)
- [UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs](https://mlsanity.com/paper/2511.01768/) (2025)
- [LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation](https://mlsanity.com/paper/2602.11007/) (2026)
- [DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR](https://mlsanity.com/paper/2604.03685/) (2026)
- [RiverMamba: A State Space Model for Global River Discharge and Flood Forecasting](https://mlsanity.com/paper/2505.22535/) (2025)
- [Diffusion-based Symbolic Music Generation with Structured State Space Models](https://mlsanity.com/paper/2507.20128/) (2025)
- [EgoMusic-driven Human Dance Motion Estimation with Skeleton Mamba](https://mlsanity.com/paper/2508.10522/) (2025)
- [Towards 3D Objectness Learning in an Open World](https://mlsanity.com/paper/2510.17686/) (2025)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)
- [Deformba: Vision State Space Model with Adaptive State Fusion](https://mlsanity.com/paper/2605.21308/) (2026)
- [BadVim: Unveiling Backdoor Threats in Visual State Space Model](https://mlsanity.com/paper/2408.11679/) (2024)
- [Instance-Adaptive Keypoint Learning with Local-to-Global Geometric Aggregation for Category-Level Object Pose Estimation](https://mlsanity.com/paper/2504.15134/) (2025)
- [Efficient LiDAR Reflectance Compression via Scanning Serialization](https://mlsanity.com/paper/2505.09433/) (2025)
- [Towards Streaming LiDAR Object Detection with Point Clouds as Egocentric Sequences](https://mlsanity.com/paper/2506.06944/) (2025)

## Abstract (arXiv)

Serialization-based methods, which serialize the 3D voxels and group them into multiple sequences before inputting to Transformers, have demonstrated their effectiveness in 3D object detection. However, serializing 3D voxels into 1D sequences will inevitably sacrifice the voxel spatial proximity. Such an issue is hard to be addressed by enlarging the group size with existing serialization-based methods due to the quadratic complexity of Transformers with feature sizes. Inspired by the recent advances of state space models (SSMs), we present a Voxel SSM, termed as Voxel Mamba, which employs a group-free strategy to serialize the whole space of voxels into a single sequence. The linear complexity of SSMs encourages our group-free design, alleviating the loss of spatial proximity of voxels. To further enhance the spatial proximity, we propose a Dual-scale SSM Block to establish a hierarchical structure, enabling a larger receptive field in the 1D serialization curve, as well as more complete local regions in 3D space. Moreover, we implicitly apply window partition under the group-free framework by positional encoding, which further enhances spatial proximity by encoding voxel positional information. Our experiments on Waymo Open Dataset and nuScenes dataset show that Voxel Mamba not only achieves higher accuracy than state-of-the-art methods, but also demonstrates significant advantages in computational efficiency.