# Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer

arXiv: https://arxiv.org/abs/2608.20659 · HTML: https://mlsanity.com/paper/2608.20659/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 155 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [SAM3D: Segment Anything in 3D Scenes](https://mlsanity.com/paper/2306.03908/) (2023)
- [Weakly Supervised 3D Open-vocabulary Segmentation](https://mlsanity.com/paper/2305.14093/) (2023)
- [SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians](https://mlsanity.com/paper/2412.10231/) (2024)
- [Segment Any 4D Gaussians](https://mlsanity.com/paper/2407.04504/) (2024)
- [CCL-LGS: Contrastive Codebook Learning for 3D Language Gaussian Splatting](https://mlsanity.com/paper/2505.20469/) (2025)
- [KeySG: Hierarchical Keyframe-Based 3D Scene Graphs](https://mlsanity.com/paper/2510.01049/) (2025)
- [Learning Segmented 3D Gaussians via Efficient Feature Unprojection for Zero-shot Neural Scene Segmentation](https://mlsanity.com/paper/2401.05925/) (2024)
- [Visibility-Aware Language Aggregation for Open-Vocabulary Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2509.05515/) (2025)
- [Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2405.15580/) (2024)
- [Pointmap Association and Piecewise-Plane Constraint for Consistent and Compact 3D Gaussian Segmentation Field](https://mlsanity.com/paper/2502.16303/) (2025)
- [4D-Editor: Interactive Object-level Editing in Dynamic Neural Radiance Fields via Semantic Distillation](https://mlsanity.com/paper/2310.16858/) (2023)
- [Efficient 3D Instance Mapping and Localization with Neural Fields](https://mlsanity.com/paper/2403.19797/) (2024)
- [PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum](https://mlsanity.com/paper/2506.23607/) (2025)
- [SAGOnline: Segment Any Gaussians Online](https://mlsanity.com/paper/2508.08219/) (2025)
- [OGScene3D: Incremental Open-Vocabulary 3D Gaussian Scene Graph Mapping for Scene Understanding](https://mlsanity.com/paper/2603.16301/) (2026)
- [ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2607.18801/) (2026)
- [PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting](https://mlsanity.com/paper/2508.00259/) (2025)
- [4D Synchronized Fields: Motion-Language Gaussian Splatting for Temporal Scene Understanding](https://mlsanity.com/paper/2603.14301/) (2026)
- [Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting](https://mlsanity.com/paper/2605.13600/) (2026)
- [CDSeg: A Renderable Gaussian Carrier for Image-to-3D Label Transfer](https://mlsanity.com/paper/2608.05482/) (2026)
- [TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation](https://mlsanity.com/paper/2602.08540/) (2026)
- [TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2605.26576/) (2026)
- [ESAM++: Efficient Online 3D Perception on the Edge](https://mlsanity.com/paper/2605.29505/) (2026)
- [OP3DSG: Open-Vocabulary Part-Aware 3D Scene Graph Generation for Real-World Environments](https://mlsanity.com/paper/2606.29786/) (2026)
- [Open-Vocabulary and Referring Segmentation for 3D Gaussians Using 2D Detectors](https://mlsanity.com/paper/2606.30638/) (2026)
- [Relation-Centric Open-Vocabulary 3D Gaussian Segmentation](https://mlsanity.com/paper/2607.01140/) (2026)

## Abstract (arXiv)

Methods that transfer predictions from two-dimensional foundation models into three-dimensional segmentation are commonly grouped by task or representation. Those groupings obscure the decisions that determine whether a system remains coherent across views: where image evidence is grounded, when observations become one identity, how semantic and granularity conflicts are handled, which information is fused, and what state survives for later queries. We introduce Lift, Associate, and Fuse (LAF), a decision-centric framework that represents a transfer system as five operators: Generate, Associate, Reconcile, Fuse, and Persist/Query. LAF defines an explicit contract for the persistent carrier---its spatial support, semantic state, identity state, uncertainty, provenance, and supported operations---and identifies the first stage at which discarded evidence becomes unrecoverable. We operationalize the framework as a structured audit protocol and apply it to 161 systems available through 7 August 2026, spanning point-, field-, Gaussian-, object-, graph-, and memory-based carriers. Representation, temporal, relational, and feed-forward stress tests required no additional analytical stage after the final confirmation pass. The resulting decision traces expose four recurring properties: association does not establish identity; carrier design fixes both the query interface and correction boundary; rendered-view, native-3D, and proposal-level evaluations are not interchangeable; and qualifiers such as training-free, real-time, open-vocabulary, and generalizable are meaningful only when attached to a stage and a complete cost ledger. LAF therefore supplies a representation-neutral method for comparing existing systems, diagnosing irreversible failures, and specifying revisable 3D perception for future agents.