# SCA-PVNet: Self-and-Cross Attention Based Aggregation of Point Cloud and Multi-View for 3D Object Retrieval

arXiv: https://arxiv.org/abs/2307.10601 · HTML: https://mlsanity.com/paper/2307.10601/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters](https://mlsanity.com/paper/1803.11527/) (2018)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [Triplet-Center Loss for Multi-View 3D Object Retrieval](https://mlsanity.com/paper/1803.06189/) (2018)
- [An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing](https://mlsanity.com/paper/1810.12000/) (2018)
- [PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition](https://mlsanity.com/paper/1808.07659/) (2018)
- [GIFT: A Real-time and Scalable 3D Shape Search Engine](https://mlsanity.com/paper/1604.01879/) (2016)
- [A Deeper Look at 3D Shape Classifiers](https://mlsanity.com/paper/1809.02560/) (2018)
- [View N-gram Network for 3D Object Retrieval](https://mlsanity.com/paper/1908.01958/) (2019)
- [3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention](https://mlsanity.com/paper/1905.07503/) (2019)
- [Angular Triplet-Center Loss for Multi-view 3D Shape Retrieval](https://mlsanity.com/paper/1811.08622/) (2018)
- [PVRNet: Point-View Relation Neural Network for 3D Shape Recognition](https://mlsanity.com/paper/1812.00333/) (2018)
- [Learning with Batch-wise Optimal Transport Loss for 3D Shape Recognition](https://mlsanity.com/paper/1903.08923/) (2019)
- [Learning Discriminative 3D Shape Representations by View Discerning Networks](https://mlsanity.com/paper/1808.03823/) (2018)
- [3DCapsule: Extending the Capsule Architecture to Classify 3D Point Clouds](https://mlsanity.com/paper/1811.02191/) (2018)

## Built on by (most cited first, 3 of 3)

- [PEVA-Net: Prompt-Enhanced View Aggregation Network for Zero/Few-Shot Multi-View 3D Shape Recognition](https://mlsanity.com/paper/2404.19168/) (2024)
- [PillarTrack:Boosting Pillar Representation for Transformer-based 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2404.07495/) (2024)
- [Enhancing Semi-Supervised Multi-View Graph Convolutional Networks via Supervised Contrastive Learning and Self-Training](https://mlsanity.com/paper/2512.13770/) (2025)

## Abstract (arXiv)

To address 3D object retrieval, substantial efforts have been made to generate highly discriminative descriptors of 3D objects represented by a single modality, e.g., voxels, point clouds or multi-view images. It is promising to leverage the complementary information from multi-modality representations of 3D objects to further improve retrieval performance. However, multi-modality 3D object retrieval is rarely developed and analyzed on large-scale datasets. In this paper, we propose self-and-cross attention based aggregation of point cloud and multi-view images (SCA-PVNet) for 3D object retrieval. With deep features extracted from point clouds and multi-view images, we design two types of feature aggregation modules, namely the In-Modality Aggregation Module (IMAM) and the Cross-Modality Aggregation Module (CMAM), for effective feature fusion. IMAM leverages a self-attention mechanism to aggregate multi-view features while CMAM exploits a cross-attention mechanism to interact point cloud features with multi-view features. The final descriptor of a 3D object for object retrieval can be obtained via concatenating the aggregated features from both modules. Extensive experiments and analysis are conducted on three datasets, ranging from small to large scale, to show the superiority of the proposed SCA-PVNet over the state-of-the-art methods.