# PillarMamba: Learning Local-Global Context for Roadside Point Cloud via Hybrid State Space Model arXiv: https://arxiv.org/abs/2505.05397 ยท HTML: https://mlsanity.com/paper/2505.05397/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba). - Builds on 32 of its 40 references that are in the map. - First posted on arXiv 8 May 2025. ## Placed in - State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/) ## Builds on (most cited first, 30 of 32) - [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016) - [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021) - [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019) - [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018) - [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017) - [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020) - [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024) - [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024) - [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018) - [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019) - [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022) - [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019) - [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023) - [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021) - [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020) - [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020) - [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024) - [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019) - [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022) - [Disentangling Monocular 3D Object Detection](https://mlsanity.com/paper/1905.12365/) (2019) - [Embracing Single Stride 3D Object Detector with Sparse Transformer](https://mlsanity.com/paper/2112.06375/) (2021) - [V2X-Seq: A Large-Scale Sequential Dataset for Vehicle-Infrastructure Cooperative Perception and Forecasting](https://mlsanity.com/paper/2305.05938/) (2023) - [Demystify Mamba in Vision: A Linear Attention Perspective](https://mlsanity.com/paper/2405.16605/) (2024) - [PillarNet: Real-Time and High-Performance Pillar-based 3D Object Detection](https://mlsanity.com/paper/2205.07403/) (2022) - [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022) - [BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection](https://mlsanity.com/paper/2303.08498/) (2023) - [PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/2305.04925/) (2023) ## Built on by (most cited first, 2 of 2) - [RoadFormer : Local-Global Feature Fusion for Road Surface Classification in Autonomous Driving](https://mlsanity.com/paper/2506.02358/) (2025) - [Defer to Plan: Adaptive Multi-Agent Fusion for End-to-End V2X Driving](https://mlsanity.com/paper/2607.19774/) (2026) ## Abstract (arXiv) Serving the Intelligent Transport System (ITS) and Vehicle-to-Everything (V2X) tasks, roadside perception has received increasing attention in recent years, as it can extend the perception range of connected vehicles and improve traffic safety. However, roadside point cloud oriented 3D object detection has not been effectively explored. To some extent, the key to the performance of a point cloud detector lies in the receptive field of the network and the ability to effectively utilize the scene context. The recent emergence of Mamba, based on State Space Model (SSM), has shaken up the traditional convolution and transformers that have long been the foundational building blocks, due to its efficient global receptive field. In this work, we introduce Mamba to pillar-based roadside point cloud perception and propose a framework based on Cross-stage State-space Group (CSG), called PillarMamba. It enhances the expressiveness of the network and achieves efficient computation through cross-stage feature fusion. However, due to the limitations of scan directions, state space model faces local connection disrupted and historical relationship forgotten. To address this, we propose the Hybrid State-space Block (HSB) to obtain the local-global context of roadside point cloud. Specifically, it enhances neighborhood connections through local convolution and preserves historical memory through residual attention. The proposed method outperforms the state-of-the-art methods on the popular large scale roadside benchmark: DAIR-V2X-I. The code will be released soon.