# LinK: Linear Kernel for LiDAR-based 3D Perception

arXiv: https://arxiv.org/abs/2303.16094 · HTML: https://mlsanity.com/paper/2303.16094/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 77 references that are in the map.
- First posted on arXiv 28 Mar 2023.

## Builds on (most cited first, 30 of 59)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)

## Built on by (most cited first, 26 of 26)

- [SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos](https://mlsanity.com/paper/2308.09244/) (2023)
- [Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection](https://mlsanity.com/paper/2406.10700/) (2024)
- [SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection](https://mlsanity.com/paper/2403.05817/) (2024)
- [HEDNet: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2310.20234/) (2023)
- [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2407.18232/) (2024)
- [DifFUSER: Diffusion Model for Robust Multi-Sensor Fusion in 3D Object Detection and BEV Segmentation](https://mlsanity.com/paper/2404.04629/) (2024)
- [LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels](https://mlsanity.com/paper/2403.15173/) (2024)
- [SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds](https://mlsanity.com/paper/2407.11569/) (2024)
- [PillarNeSt: Embracing Backbone Scaling and Pretraining for Pillar-based 3D Object Detection](https://mlsanity.com/paper/2311.17770/) (2023)
- [CR3DT: Camera-RADAR Fusion for 3D Detection and Tracking](https://mlsanity.com/paper/2403.15313/) (2024)
- [A Point-Based Approach to Efficient LiDAR Multi-Task Perception](https://mlsanity.com/paper/2404.12798/) (2024)
- [FSHNet: Fully Sparse Hybrid Network for 3D Object Detection](https://mlsanity.com/paper/2506.03714/) (2025)
- [Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness](https://mlsanity.com/paper/2407.08972/) (2024)
- [Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations](https://mlsanity.com/paper/2410.08049/) (2024)
- [NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentation](https://mlsanity.com/paper/2505.24634/) (2025)
- [Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection](https://mlsanity.com/paper/2304.02867/) (2023)
- [AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environments](https://mlsanity.com/paper/2603.25494/) (2026)
- [Large receptive field strategy and important feature extraction strategy in 3D object detection](https://mlsanity.com/paper/2401.11913/) (2024)
- [Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning](https://mlsanity.com/paper/2411.07742/) (2024)
- [Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2503.10579/) (2025)
- [Power of Cooperative Supervision: Multiple Teachers Framework for Enhanced 3D Semi-Supervised Object Detection](https://mlsanity.com/paper/2405.20720/) (2024)
- [SFMNet: Sparse Focal Modulation for 3D Object Detection](https://mlsanity.com/paper/2503.12093/) (2025)
- [FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices](https://mlsanity.com/paper/2506.04499/) (2025)
- [WinMamba: Multi-Scale Shifted Windows in State Space Model for 3D Object Detection](https://mlsanity.com/paper/2511.13138/) (2025)
- [RayMamba: Ray-Aligned Serialization for Long-Range 3D Object Detection](https://mlsanity.com/paper/2604.02903/) (2026)
- [PointLAM: Local Attentive Mamba for Efficient Point-based 3D Object Detection](https://mlsanity.com/paper/2609.21780/) (2026)

## Abstract (arXiv)

Extending the success of 2D Large Kernel to 3D perception is challenging due to: 1. the cubically-increasing overhead in processing 3D data; 2. the optimization difficulties from data scarcity and sparsity. Previous work has taken the first step to scale up the kernel size from 3x3x3 to 7x7x7 by introducing block-shared weights. However, to reduce the feature variations within a block, it only employs modest block size and fails to achieve larger kernels like the 21x21x21. To address this issue, we propose a new method, called LinK, to achieve a wider-range perception receptive field in a convolution-like manner with two core designs. The first is to replace the static kernel matrix with a linear kernel generator, which adaptively provides weights only for non-empty voxels. The second is to reuse the pre-computed aggregation results in the overlapped blocks to reduce computation complexity. The proposed method successfully enables each voxel to perceive context within a range of 21x21x21. Extensive experiments on two basic perception tasks, 3D object detection and 3D semantic segmentation, demonstrate the effectiveness of our method. Notably, we rank 1st on the public leaderboard of the 3D detection benchmark of nuScenes (LiDAR track), by simply incorporating a LinK-based backbone into the basic detector, CenterPoint. We also boost the strong segmentation baseline's mIoU with 2.7% in the SemanticKITTI test set. Code is available at https://github.com/MCG-NJU/LinK.