# 3D CoCa v2: Contrastive Learners with Test-Time Search for Generalizable Spatial Intelligence

arXiv: https://arxiv.org/abs/2601.06496 · HTML: https://mlsanity.com/paper/2601.06496/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 52 references that are in the map.
- First posted on arXiv 10 Jan 2026.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Scan2Cap: Context-aware Dense Captioning in RGB-D Scans](https://mlsanity.com/paper/2012.02206/) (2020)
- [Unsupervised Point Cloud Pre-Training via Occlusion Completion](https://mlsanity.com/paper/2010.01089/) (2020)
- [End-to-End 3D Dense Captioning with Vote2Cap-DETR](https://mlsanity.com/paper/2301.02508/) (2023)
- [X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning](https://mlsanity.com/paper/2203.00843/) (2022)
- [UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding](https://mlsanity.com/paper/2212.00836/) (2022)
- [3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding](https://mlsanity.com/paper/2507.23478/) (2025)
- [MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenes](https://mlsanity.com/paper/2203.05203/) (2022)
- [SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos](https://mlsanity.com/paper/2104.11452/) (2021)
- [Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds](https://mlsanity.com/paper/2204.10688/) (2022)
- [TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes](https://mlsanity.com/paper/2403.19589/) (2024)
- [Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning](https://mlsanity.com/paper/2309.02999/) (2023)
- [Evaluation of Best-of-N Sampling Strategies for Language Model Alignment](https://mlsanity.com/paper/2502.12668/) (2025)
- [Nav-R1: Reasoning and Navigation in Embodied Scenes](https://mlsanity.com/paper/2509.10884/) (2025)
- [VLA-R1: Enhancing Reasoning in Vision-Language-Action Models](https://mlsanity.com/paper/2510.01623/) (2025)
- [AgentRM: Enhancing Agent Generalization with Reward Modeling](https://mlsanity.com/paper/2502.18407/) (2025)
- [ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models](https://mlsanity.com/paper/2505.16517/) (2025)
- [TightCap: 3D Human Shape Capture with Clothing Tightness Field](https://mlsanity.com/paper/1904.02601/) (2019)
- [Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding](https://mlsanity.com/paper/2305.10714/) (2023)
- [Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models](https://mlsanity.com/paper/2307.14971/) (2023)
- [Hazards in Daily Life? Enabling Robots to Proactively Detect and Resolve Anomalies](https://mlsanity.com/paper/2411.00781/) (2024)
- [DC-Scene: Data-Centric Learning for 3D Scene Understanding](https://mlsanity.com/paper/2505.15232/) (2025)
- [MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robots](https://mlsanity.com/paper/2511.17889/) (2025)

## Built on by (most cited first, 3 of 3)

- [UniMesh: Unifying 3D Mesh Understanding and Generation](https://mlsanity.com/paper/2604.17472/) (2026)
- [ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning](https://mlsanity.com/paper/2607.17599/) (2026)
- [MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control](https://mlsanity.com/paper/2609.06251/) (2026)

## Abstract (arXiv)

Spatial intelligence refers to the ability to perceive, reason about, and describe objects and their relationships within three-dimensional environments, forming a foundation for embodied perception and scene understanding. 3D captioning aims to describe 3D scenes in natural language; however, it remains challenging due to the sparsity and irregularity of point clouds and, more critically, the weak grounding and limited out-of-distribution (OOD) generalization of existing captioners across drastically different environments, including indoor and outdoor 3D scenes. To address this challenge, we propose 3D CoCa v2, a generalizable 3D captioning framework that unifies contrastive vision-language learning with 3D caption generation and further improves robustness via test-time search (TTS) without updating the captioner parameters. 3D CoCa v2 builds on a frozen CLIP-based semantic prior, a spatially-aware 3D scene encoder for geometry, and a multimodal decoder jointly optimized with contrastive and captioning objectives, avoiding external detectors or handcrafted proposals. At inference, TTS produces diverse caption candidates and performs reward-guided selection using a compact scene summary. Experiments show improvements over 3D CoCa of +1.50 CIDEr@0.5IoU on ScanRefer and +1.61 CIDEr@0.5IoU on Nr3D, and +3.8 CIDEr@0.25 in zero-shot OOD evaluation on TOD3Cap. Code will be released at https://github.com/AIGeeksGroup/3DCoCav2.