mlsanityData to 7 Oct 2026

TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval

arXiv 2201.07366 ↗First posted 19 Jan 2022cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

Text-to-shape retrieval is an increasingly relevant problem with the growth of 3D shape data. Recent work on contrastive losses for learning joint embeddings over multimodal data has been successful at tasks such as retrieval and classification. Thus far, work on joint representation learning for 3D shapes and text has focused on improving embeddings through modeling of complex attention between representations, or multi-task learning. We propose a trimodal learning scheme over text, multi-view images and 3D shape voxels, and show that with large batch contrastive learning we achieve good performance on text-to-shape retrieval without complex attention mechanisms or losses. Our experiments serve as a foundation for follow-up work on building trimodal embeddings for text-image-shape.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 48 of its 85 references that are in the map. First posted on arXiv 19 Jan 2022.

Builds on

48 of 85 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
36 more
  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  8. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  9. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  10. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  11. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  12. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  13. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  14. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  15. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  16. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  17. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  18. 2020Self-Supervised MultiModal Versatile Networkscited by 276
  19. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  20. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  21. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 197
  22. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  23. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  24. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 115
  25. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  26. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  27. 2017Colors in Context: A Pragmatic Neural Model for Grounded Language Understandingcited by 57
  28. 2019ShapeGlot: Learning Language for Shape Differentiationcited by 54
  29. 2021Contrastive Multimodal Fusion with TupleInfoNCEcited by 43
  30. 2021Language Grounding with 3D Objectscited by 43
  31. 2021Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysiscited by 41
  32. 2019ShapeCaptioner: Generative Caption Network for 3D Shapes by Learning a Mapping from Parts Detected in Multiple Views to Sentencescited by 30
  33. 2018Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequencescited by 26
  34. 2017Learning by Association - A versatile semi-supervised training method for neural networkscited by 19
  35. 2021Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Visioncited by 18
  36. 2021Parts2Words: Learning Joint Embedding of Point Clouds and Texts by Bidirectional Matching between Parts and Wordscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 6; 2026: 4.

18 papers.

Sort:

By area:

  1. 2024CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scalecited by 18
  2. 2024Duoduo CLIP: Efficient 3D Understanding with Multi-View Imagescited by 12
  3. 20253DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scalecited by 12
  4. 2025SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentationcited by 9
  5. 2024PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representationcited by 8
  6. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  7. 2024COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrievalcited by 6
  8. 2024A Survey of Multimodal Composite Editing and Retrievalcited by 5
  9. 2025Enhanced Cross-modal 3D Retrieval via Tri-modal Reconstructioncited by 5
  10. 2025A Principled Framework for Multi-View Contrastive Learningcited by 4
  11. 2023MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representationscited by 3
  12. 2025MonSTeR: a Unified Model for Motion, Scene, Text Retrievalcited by 2
6 more
  1. 2024N-Modal Contrastive Losses with Applications to Social Media Data in Trimodal Spacecited by 1
  2. 2026TIGaussian: Disentangle Gaussians for Spatial-Awared Text-Image-3D Alignmentcited by 1
  3. 2023Text2shape Deep Retrieval Model: Generating Initial Cases for Mechanical Part Redesign under the Context of Case-Based Reasoningcited by 0
  4. 2026RI-Mamba: Rotation-Invariant Mamba for Robust Text-to-Shape Retrievalcited by 0
  5. 2026Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshescited by 0
  6. 2026Text-to-CAD Retrieval: a Strong Baselinecited by 0