mlsanityData to 7 Oct 2026

Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views

arXiv 2606.29513 ↗First posted 28 Jun 2026cs.CV cs.GRcited by 1

Not on the map. Why?

Abstract (arXiv)

A 3D scene is understood through its objects, not the primitives that compose them. Yet feed-forward reconstruction methods output dense, unstructured sets of points or Gaussians, leaving object-level structure to be recovered after the fact. We propose a feed-forward framework that decomposes a scene into instance-structured 3D token groups directly from unposed multi-view images -- compact object-centric units from which reconstruction, segmentation, and manipulation all follow. Each token group pairs an instance token capturing entity-level identity with anchor tokens that encode local geometry and appearance, which are decoded into a set of 3D Gaussians. This two-level factorization decouples object identity from local appearance, making object instances a native interface of the representation rather than a derived product. The token groups are learned through differentiable rendering with joint reconstruction and segmentation supervision, requiring no 3D annotations. Our feed-forward model surpasses per-scene optimization baselines in class-agnostic instance segmentation while remaining competitive in novel view synthesis. Beyond these metrics, the same token groups directly unlock instance-level scene editing -- removing, translating, or inserting objects by operating on their groups -- as well as efficient open-vocabulary 3D instance retrieval, where retrieval complexity scales with the number of instances rather than primitives.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 36 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

34 of 36 references.

References · 34

  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  10. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  11. 2020Object-Centric Learning with Slot Attentioncited by 865
  12. 2022Language-driven Semantic Segmentationcited by 711
22 more
  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  2. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  3. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
  4. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  5. 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 279
  6. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  7. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  8. 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 171
  9. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  10. 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
  11. 2024LUDVIG: Learning-Free Uplifting of 2D Visual Features to Gaussian Splatting Scenescited by 36
  12. 2025IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstructioncited by 26
  13. 2025Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Imagescited by 25
  14. 2025ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splattingcited by 24
  15. 2025Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentationcited by 23
  16. 2025PanSt3R: Multi-view Consistent Panoptic Segmentationcited by 19
  17. 2025C3G: Learning Compact 3D Representations with 2K Gaussianscited by 18
  18. 2024Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Trackingcited by 14
  19. 2025Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracingcited by 13
  20. 2022kMaX-DeepLab: k-means Mask Transformercited by 12
  21. 2024SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fieldscited by 12
  22. 2026Scene-Agnostic Object-Centric Representation Learning for 3D Gaussian Splattingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.