# LION: Linear Group RNN for 3D Object Detection in Point Clouds

arXiv: https://arxiv.org/abs/2407.18232 · HTML: https://mlsanity.com/paper/2407.18232/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 69 of its 77 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/)

## Builds on (most cited first, 30 of 69)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://mlsanity.com/paper/2312.06635/) (2023)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [xLSTM: Extended Long Short-Term Memory](https://mlsanity.com/paper/2405.04517/) (2024)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models](https://mlsanity.com/paper/2402.19427/) (2024)

## Built on by (most cited first, 30 of 38)

- [Spatial Understanding from Videos: Structured Prompts Meet Simulation Data](https://mlsanity.com/paper/2506.03642/) (2025)
- [Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model](https://mlsanity.com/paper/2503.16282/) (2025)
- [DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction](https://mlsanity.com/paper/2409.19972/) (2024)
- [UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2503.12009/) (2025)
- [FSHNet: Fully Sparse Hybrid Network for 3D Object Detection](https://mlsanity.com/paper/2506.03714/) (2025)
- [A Survey of RWKV](https://mlsanity.com/paper/2412.14847/) (2024)
- [Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation](https://mlsanity.com/paper/2409.11018/) (2024)
- [TARS: Traffic-Aware Radar Scene Flow Estimation](https://mlsanity.com/paper/2503.10210/) (2025)
- [MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection](https://mlsanity.com/paper/2507.04369/) (2025)
- [UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs](https://mlsanity.com/paper/2511.01768/) (2025)
- [Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking](https://mlsanity.com/paper/2602.23172/) (2026)
- [DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR](https://mlsanity.com/paper/2604.03685/) (2026)
- [PPT: Pretraining with Pseudo-Labeled Trajectories for Motion Forecasting](https://mlsanity.com/paper/2412.06491/) (2024)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)
- [SEGT: A General Spatial Expansion Group Transformer for nuScenes Lidar-based Object Detection Task](https://mlsanity.com/paper/2412.09658/) (2024)
- [POD: Predictive Object Detection with Single-Frame FMCW LiDAR Point Cloud](https://mlsanity.com/paper/2504.05649/) (2025)
- [PAN: Pillars-Attention-Based Network for 3D Object Detection](https://mlsanity.com/paper/2509.15935/) (2025)
- [Comprehensive Assessment of LiDAR Evaluation Metrics: A Comparative Study Using Simulated and Real Data](https://mlsanity.com/paper/2511.02994/) (2025)
- [GeoTeacher: Geometry-Guided Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2512.23147/) (2025)
- [On the Feasibility and Opportunity of Autoregressive 3D Object Detection](https://mlsanity.com/paper/2603.07985/) (2026)
- [RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception](https://mlsanity.com/paper/2606.31895/) (2026)
- [SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video](https://mlsanity.com/paper/2607.01784/) (2026)
- [Dual-Domain Homogeneous Fusion with Cross-Modal Mamba and Progressive Decoder for 3D Object Detection](https://mlsanity.com/paper/2503.08992/) (2025)
- [An Efficient LiDAR-Camera Fusion Network for Multi-Class 3D Dynamic Object Detection and Trajectory Prediction](https://mlsanity.com/paper/2504.13647/) (2025)
- [FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices](https://mlsanity.com/paper/2506.04499/) (2025)
- [DySS: Dynamic Queries and State-Space Learning for Efficient 3D Object Detection from Multi-Camera Videos](https://mlsanity.com/paper/2506.10242/) (2025)
- [HybridTM: Combining Transformer and Mamba for 3D Semantic Segmentation](https://mlsanity.com/paper/2507.18575/) (2025)
- [Voxel Densification for Serialized 3D Object Detection: Mitigating Sparsity via Pre-serialization Expansion](https://mlsanity.com/paper/2508.16069/) (2025)
- [PointSlice: Accurate and Efficient Slice-Based Representation for 3D Object Detection from Point Clouds](https://mlsanity.com/paper/2509.01487/) (2025)
- [Fore-Mamba3D: Mamba-based Foreground-Enhanced Encoding for 3D Object Detection](https://mlsanity.com/paper/2602.19536/) (2026)

## Abstract (arXiv)

The benefit of transformers in large-scale 3D point cloud perception tasks, such as 3D object detection, is limited by their quadratic computation cost when modeling long-range relationships. In contrast, linear RNNs have low computational complexity and are suitable for long-range modeling. Toward this goal, we propose a simple and effective window-based framework built on LInear grOup RNN (i.e., perform linear RNN for grouped features) for accurate 3D object detection, called LION. The key property is to allow sufficient feature interaction in a much larger group than transformer-based methods. However, effectively applying linear group RNN to 3D object detection in highly sparse point clouds is not trivial due to its limitation in handling spatial modeling. To tackle this problem, we simply introduce a 3D spatial feature descriptor and integrate it into the linear group RNN operators to enhance their spatial features rather than blindly increasing the number of scanning orders for voxel features. To further address the challenge in highly sparse point clouds, we propose a 3D voxel generation strategy to densify foreground features thanks to linear group RNN as a natural property of auto-regressive models. Extensive experiments verify the effectiveness of the proposed components and the generalization of our LION on different linear group RNN operators including Mamba, RWKV, and RetNet. Furthermore, it is worth mentioning that our LION-Mamba achieves state-of-the-art on Waymo, nuScenes, Argoverse V2, and ONCE dataset. Last but not least, our method supports kinds of advanced linear RNN operators (e.g., RetNet, RWKV, Mamba, xLSTM and TTT) on small but popular KITTI dataset for a quick experience with our linear RNN-based framework.