# V2-SAM: Marrying SAM2 with Multi-Prompt Experts for Cross-View Object Correspondence

arXiv: https://arxiv.org/abs/2511.20886 · HTML: https://mlsanity.com/paper/2511.20886/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 68 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 30 of 55)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos](https://mlsanity.com/paper/2501.04001/) (2025)
- [FastMoE: A Fast Mixture-of-Expert Training System](https://mlsanity.com/paper/2103.13262/) (2021)
- [Deep Geometric Prior for Surface Reconstruction](https://mlsanity.com/paper/1811.10943/) (2018)
- [PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model](https://mlsanity.com/paper/2403.14598/) (2024)
- [Locate then Segment: A Strong Pipeline for Referring Image Segmentation](https://mlsanity.com/paper/2103.16284/) (2021)
- [Multi-view PointNet for 3D Scene Understanding](https://mlsanity.com/paper/1909.13603/) (2019)
- [Siamese Masked Autoencoders](https://mlsanity.com/paper/2305.14344/) (2023)
- [GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulation](https://mlsanity.com/paper/1906.12192/) (2019)
- [VRP-SAM: SAM with Visual Reference Prompt](https://mlsanity.com/paper/2402.17726/) (2024)
- [Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning](https://mlsanity.com/paper/2503.00513/) (2025)
- [Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition](https://mlsanity.com/paper/2010.09982/) (2020)
- [Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community](https://mlsanity.com/paper/2408.09110/) (2024)
- [HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions](https://mlsanity.com/paper/2308.01477/) (2023)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance](https://mlsanity.com/paper/2510.24711/) (2025)
- [Enhance Then Search: An Augmentation-Search Strategy with Foundation Models for Cross-Domain Few-Shot Object Detection](https://mlsanity.com/paper/2504.04517/) (2025)

## Built on by (most cited first, 14 of 14)

- [EgoSound: Benchmarking Sound Understanding in Egocentric Videos](https://mlsanity.com/paper/2602.14122/) (2026)
- [PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learning](https://mlsanity.com/paper/2602.21992/) (2026)
- [Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence](https://mlsanity.com/paper/2607.12477/) (2026)
- [ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding](https://mlsanity.com/paper/2607.28312/) (2026)
- [From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation](https://mlsanity.com/paper/2604.13793/) (2026)
- [CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception](https://mlsanity.com/paper/2605.23655/) (2026)
- [Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning](https://mlsanity.com/paper/2605.26761/) (2026)
- [Adaptive Vision-Language Grasping via Composable Foundation Priors and Generalizable Grasp Synthesis](https://mlsanity.com/paper/2609.04096/) (2026)
- [SAM-V: Geometry-Aware Segment Anything for Multi-View Instance Segmentation](https://mlsanity.com/paper/2609.25490/) (2026)
- [ReconMIL: Synergizing Latent Space Reconstruction with Bi-Stream Mamba for Whole Slide Image Analysis](https://mlsanity.com/paper/2603.19925/) (2026)
- [Learning Cross-View Semantic Priors for Single-Reference Unseen Object Pose Estimation](https://mlsanity.com/paper/2606.22076/) (2026)
- [The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering](https://mlsanity.com/paper/2608.04589/) (2026)
- [EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos](https://mlsanity.com/paper/2610.17128/) (2026)
- [EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactions](https://mlsanity.com/paper/2610.18499/) (2026)

## Abstract (arXiv)

Cross-view object correspondence, exemplified by the representative task of ego-exo object correspondence, aims to establish consistent associations of the same object across different viewpoints (e.g., egocentric and exocentric). This task poses significant challenges due to drastic viewpoint and appearance variations, making existing segmentation models, such as SAM2, difficult to apply directly. To address this, we present V2-SAM, a unified cross-view object correspondence framework that adapts SAM2 from single-view segmentation to cross-view correspondence through two complementary prompt generators. Specifically, the Cross-View Anchor Prompt Generator (V2-Anchor), built upon DINOv3 features, establishes geometry-aware correspondences and, for the first time, enables coordinate-based prompting for SAM2 in cross-view scenarios, while the Cross-View Visual Prompt Generator (V2-Visual) enhances appearance-guided cues via a novel visual prompt matcher that aligns ego-exo representations from both feature and structural perspectives. To effectively exploit the strengths of both prompts, we further adopt a multi-expert design and introduce a Post-hoc Cyclic Consistency Selector (PCCS) that adaptively selects the most reliable expert based on cyclic consistency. Extensive experiments validate the effectiveness of V2-SAM, achieving new state-of-the-art performance on Ego-Exo4D (ego-exo object correspondence), DAVIS-2017 (video object tracking), and HANDAL-X (robotic-ready cross-view correspondence).