# COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval

arXiv: https://arxiv.org/abs/2405.04103 · HTML: https://mlsanity.com/paper/2405.04103/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations](https://mlsanity.com/paper/2111.13152/) (2021)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training](https://mlsanity.com/paper/2210.01055/) (2022)
- [MVSTER: Epipolar Transformer for Efficient Multi-View Stereo](https://mlsanity.com/paper/2204.07346/) (2022)
- [RUST: Latent Neural Scene Representations from Unposed Imagery](https://mlsanity.com/paper/2211.14306/) (2022)
- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences](https://mlsanity.com/paper/1811.02745/) (2018)
- [TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval](https://mlsanity.com/paper/2201.07366/) (2022)
- [Parts2Words: Learning Joint Embedding of Point Clouds and Texts by Bidirectional Matching between Parts and Words](https://mlsanity.com/paper/2107.01872/) (2021)
- [Uni3DL: Unified Model for 3D and Language Understanding](https://mlsanity.com/paper/2312.03026/) (2023)
- [Hard Example Generation by Texture Synthesis for Cross-domain Shape Similarity Learning](https://mlsanity.com/paper/2010.12238/) (2020)

## Built on by (most cited first, 6 of 6)

- [3DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding](https://mlsanity.com/paper/2506.01946/) (2025)
- [3DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scale](https://mlsanity.com/paper/2511.13211/) (2025)
- [SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentation](https://mlsanity.com/paper/2502.19128/) (2025)
- [Enhanced Cross-modal 3D Retrieval via Tri-modal Reconstruction](https://mlsanity.com/paper/2504.01476/) (2025)
- [MetaFind: Scene-Aware 3D Asset Retrieval for Coherent Metaverse Scene Generation](https://mlsanity.com/paper/2510.04057/) (2025)
- [RI-Mamba: Rotation-Invariant Mamba for Robust Text-to-Shape Retrieval](https://mlsanity.com/paper/2602.11673/) (2026)

## Abstract (arXiv)

In this paper, we investigate an open research task of cross-modal retrieval between 3D shapes and textual descriptions. Previous approaches mainly rely on point cloud encoders for feature extraction, which may ignore key inherent features of 3D shapes, including depth, spatial hierarchy, geometric continuity, etc. To address this issue, we propose COM3D, making the first attempt to exploit the cross-view correspondence and cross-modal mining to enhance the retrieval performance. Notably, we augment the 3D features through a scene representation transformer, to generate cross-view correspondence features of 3D shapes, which enrich the inherent features and enhance their compatibility with text matching. Furthermore, we propose to optimize the cross-modal matching process based on the semi-hard negative example mining method, in an attempt to improve the learning efficiency. Extensive quantitative and qualitative experiments demonstrate the superiority of our proposed COM3D, achieving state-of-the-art results on the Text2Shape dataset.