# End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds

arXiv: https://arxiv.org/abs/2003.05855 · HTML: https://mlsanity.com/paper/2003.05855/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2020.

## Built on by (most cited first, 30 of 47)

- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency](https://mlsanity.com/paper/2103.05465/) (2021)
- [Lepard: Learning partial point cloud matching in rigid and deformable scenes](https://mlsanity.com/paper/2111.12591/) (2021)
- [Equivariant Point Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2103.14147/) (2021)
- [CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognition](https://mlsanity.com/paper/2303.11313/) (2023)
- [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021)
- [Deep Learning on Implicit Neural Representations of Shapes](https://mlsanity.com/paper/2302.05438/) (2023)
- [FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation models](https://mlsanity.com/paper/2312.00947/) (2023)
- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [Approximate Differentiable Rendering with Algebraic Surfaces](https://mlsanity.com/paper/2207.10606/) (2022)
- [When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models](https://mlsanity.com/paper/2405.10255/) (2024)
- [Learning general and distinctive 3D local deep descriptors for point cloud registration](https://mlsanity.com/paper/2105.10382/) (2021)
- [Bootstrap Your Own Correspondences](https://mlsanity.com/paper/2106.00677/) (2021)
- [DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classification](https://mlsanity.com/paper/2305.15957/) (2023)
- [Rethinking the compositionality of point clouds through regularization in the hyperbolic space](https://mlsanity.com/paper/2209.10318/) (2022)
- [E2PN: Efficient SE(3)-Equivariant Point Network](https://mlsanity.com/paper/2206.05398/) (2022)
- [SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence](https://mlsanity.com/paper/2209.07806/) (2022)
- [JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fields](https://mlsanity.com/paper/2303.15427/) (2023)
- [IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration](https://mlsanity.com/paper/2111.09624/) (2021)
- [GPSFormer: A Global Perception and Local Structure Fitting-based Transformer for Point Cloud Understanding](https://mlsanity.com/paper/2407.13519/) (2024)
- [3D Point Cloud Registration with Multi-Scale Architecture and Unsupervised Transfer Learning](https://mlsanity.com/paper/2103.14533/) (2021)
- [LighTN: Light-weight Transformer Network for Performance-overhead Tradeoff in Point Cloud Downsampling](https://mlsanity.com/paper/2202.06263/) (2022)
- [WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration](https://mlsanity.com/paper/2108.02740/) (2021)
- [Leveraging Large-Scale Pretrained Vision Foundation Models for Label-Efficient 3D Point Cloud Segmentation](https://mlsanity.com/paper/2311.01989/) (2023)
- [Generalizable Local Feature Pre-training for Deformable Shape Analysis](https://mlsanity.com/paper/2303.15104/) (2023)
- [Projection-based Point Convolution for Efficient Point Cloud Segmentation](https://mlsanity.com/paper/2202.01991/) (2022)
- [Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision](https://mlsanity.com/paper/2208.05110/) (2022)
- [Efficient Spiking Point Mamba for Point Cloud Analysis](https://mlsanity.com/paper/2504.14371/) (2025)
- [Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2210.11237/) (2022)
- [Learning a Task-specific Descriptor for Robust Matching of 3D Point Clouds](https://mlsanity.com/paper/2210.14899/) (2022)

## Abstract (arXiv)

In this work, we propose an end-to-end framework to learn local multi-view descriptors for 3D point clouds. To adopt a similar multi-view representation, existing studies use hand-crafted viewpoints for rendering in a preprocessing stage, which is detached from the subsequent descriptor learning stage. In our framework, we integrate the multi-view rendering into neural networks by using a differentiable renderer, which allows the viewpoints to be optimizable parameters for capturing more informative local context of interest points. To obtain discriminative descriptors, we also design a soft-view pooling module to attentively fuse convolutional features across views. Extensive experiments on existing 3D registration benchmarks show that our method outperforms existing local descriptors both quantitatively and qualitatively.