# A Unified Query-based Paradigm for Point Cloud Understanding

arXiv: https://arxiv.org/abs/2203.01252 · HTML: https://mlsanity.com/paper/2203.01252/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 75 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)

## Built on by (most cited first, 30 of 32)

- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)
- [MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying](https://mlsanity.com/paper/2306.17770/) (2023)
- [Visual Point Cloud Forecasting enables Scalable Autonomous Driving](https://mlsanity.com/paper/2312.17655/) (2023)
- [UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation](https://mlsanity.com/paper/2308.07732/) (2023)
- [Unifying 3D Vision-Language Understanding via Promptable Queries](https://mlsanity.com/paper/2405.11442/) (2024)
- [Fully Sparse 3D Occupancy Prediction](https://mlsanity.com/paper/2312.17118/) (2023)
- [GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds](https://mlsanity.com/paper/2212.03010/) (2022)
- [V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection](https://mlsanity.com/paper/2308.04409/) (2023)
- [What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models](https://mlsanity.com/paper/2512.03422/) (2025)
- [LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers](https://mlsanity.com/paper/2210.12755/) (2022)
- [PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer](https://mlsanity.com/paper/2305.06621/) (2023)
- [Point Cloud Classification Using Content-based Transformer via Clustering in Feature Space](https://mlsanity.com/paper/2303.04599/) (2023)
- [LitePT: Lighter Yet Stronger Point Transformer](https://mlsanity.com/paper/2512.13689/) (2025)
- [KPConvX: Modernizing Kernel Point Convolution with Kernel Attention](https://mlsanity.com/paper/2405.13194/) (2024)
- [Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understanding](https://mlsanity.com/paper/2305.05026/) (2023)
- [Unleash the Potential of Image Branch for Cross-modal 3D Object Detection](https://mlsanity.com/paper/2301.09077/) (2023)
- [Uni3DL: Unified Model for 3D and Language Understanding](https://mlsanity.com/paper/2312.03026/) (2023)
- [Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2308.09314/) (2023)
- [Improving Distant 3D Object Detection Using 2D Box Supervision](https://mlsanity.com/paper/2403.09230/) (2024)
- [MsSVT++: Mixed-scale Sparse Voxel Transformer with Center Voting for 3D Object Detection](https://mlsanity.com/paper/2401.11718/) (2024)
- [3D Harmonic Loss: Towards Task-consistent and Time-friendly 3D Object Detection on Edge for V2X Orchestration](https://mlsanity.com/paper/2211.03407/) (2022)
- [ThermRad: A Multi-modal Dataset for Robust 3D Object Detection under Challenging Conditions](https://mlsanity.com/paper/2308.10161/) (2023)
- [Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2403.06166/) (2024)
- [Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach](https://mlsanity.com/paper/2307.12049/) (2023)
- [SGCCNet: Single-Stage 3D Object Detector With Saliency-Guided Data Augmentation and Confidence Correction Mechanism](https://mlsanity.com/paper/2407.01239/) (2024)
- [FGU3R: Fine-Grained Fusion via Unified 3D Representation for Multimodal 3D Object Detection](https://mlsanity.com/paper/2501.04373/) (2025)
- [PDM-SSD: Single-Stage Three-Dimensional Object Detector With Point Dilation](https://mlsanity.com/paper/2502.07822/) (2025)
- [Vision Generalist Model: A Survey](https://mlsanity.com/paper/2506.09954/) (2025)
- [Point Virtual Transformer](https://mlsanity.com/paper/2602.06406/) (2026)

## Abstract (arXiv)

3D point cloud understanding is an important component in autonomous driving and robotics. In this paper, we present a novel Embedding-Querying paradigm (EQ- Paradigm) for 3D understanding tasks including detection, segmentation, and classification. EQ-Paradigm is a unified paradigm that enables the combination of any existing 3D backbone architectures with different task heads. Under the EQ-Paradigm, the input is firstly encoded in the embedding stage with an arbitrary feature extraction architecture, which is independent of tasks and heads. Then, the querying stage enables the encoded features to be applicable for diverse task heads. This is achieved by introducing an intermediate representation, i.e., Q-representation, in the querying stage to serve as a bridge between the embedding stage and task heads. We design a novel Q- Net as the querying stage network. Extensive experimental results on various 3D tasks, including object detection, semantic segmentation and shape classification, show that EQ-Paradigm in tandem with Q-Net is a general and effective pipeline, which enables a flexible collaboration of backbones and heads, and further boosts the performance of the state-of-the-art methods. Codes and models are available at https://github.com/dvlab-research/DeepVision3D.