# Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images

arXiv: https://arxiv.org/abs/2508.03643 · HTML: https://mlsanity.com/paper/2508.03643/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 47 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [DepthSplat: Connecting Gaussian Splatting and Depth](https://mlsanity.com/paper/2410.13862/) (2024)
- [No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images](https://mlsanity.com/paper/2410.24207/) (2024)
- [Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image](https://mlsanity.com/paper/2012.09855/) (2020)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs](https://mlsanity.com/paper/2408.13912/) (2024)
- [MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2405.12218/) (2024)
- [Large Spatial Model: End-to-end Unposed Images to Semantic 3D](https://mlsanity.com/paper/2410.18956/) (2024)
- [GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding](https://mlsanity.com/paper/2412.13193/) (2024)
- [SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians](https://mlsanity.com/paper/2412.10231/) (2024)

## Built on by (most cited first, 26 of 26)

- [IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction](https://mlsanity.com/paper/2510.22706/) (2025)
- [UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generation](https://mlsanity.com/paper/2509.25079/) (2025)
- [Spa3R: Predictive Spatial Field Modeling for 3D Visual Reasoning](https://mlsanity.com/paper/2602.21186/) (2026)
- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [FLEG: Feed-Forward Language Embedded Gaussian Splatting from Any Views via Compact Semantic Representation](https://mlsanity.com/paper/2512.17541/) (2025)
- [3AM: 3egment Anything with Geometric Consistency in Videos](https://mlsanity.com/paper/2601.08831/) (2026)
- [FAST3DIS: Feed-forward Anchored Scene Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2603.25993/) (2026)
- [EVolSplat4D: Efficient Volume-based Gaussian Splatting for 4D Urban Scene Synthesis](https://mlsanity.com/paper/2601.15951/) (2026)
- [SegVGGT: Joint 3D Reconstruction and Instance Segmentation from Multi-View Images](https://mlsanity.com/paper/2603.19926/) (2026)
- [SLARM: Streaming and Language-Aligned Reconstruction Model for Dynamic Scenes](https://mlsanity.com/paper/2603.22893/) (2026)
- [FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views](https://mlsanity.com/paper/2604.09862/) (2026)
- [EPS3D: End-to-End Feed-Forward 3D Panoptic Segmentation](https://mlsanity.com/paper/2606.08980/) (2026)
- [StructSplat: Generalizable 3D Gaussian Splatting from Uncalibrated Sparse Views](https://mlsanity.com/paper/2606.28321/) (2026)
- [IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer](https://mlsanity.com/paper/2607.19228/) (2026)
- [Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Images](https://mlsanity.com/paper/2604.10573/) (2026)
- [Pano3D: Unified 3D Reconstruction and Panoptic Segmentation](https://mlsanity.com/paper/2606.14307/) (2026)
- [Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views](https://mlsanity.com/paper/2606.29513/) (2026)
- [Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images](https://mlsanity.com/paper/2607.01633/) (2026)
- [NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction](https://mlsanity.com/paper/2607.07168/) (2026)
- [S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction](https://mlsanity.com/paper/2603.14232/) (2026)
- [UniSem: Generalizable Semantic 3D Reconstruction from Sparse Unposed Images](https://mlsanity.com/paper/2603.17519/) (2026)
- [GroupForward: Building Referable 3D Scenes via Instance-Grouped Feed-Forward Gaussian Splatting](https://mlsanity.com/paper/2608.17535/) (2026)
- [DDMS: Discriminative Distillation of Multi-view Foundational Features into Single-view Models](https://mlsanity.com/paper/2608.23850/) (2026)
- [Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction](https://mlsanity.com/paper/2609.04201/) (2026)
- [S4VY: Segment Anything in Feed-Forward 4D Visual Geometry](https://mlsanity.com/paper/2609.36875/) (2026)
- [RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation](https://mlsanity.com/paper/2610.00970/) (2026)

## Abstract (arXiv)

Reconstructing and semantically interpreting 3D scenes from sparse 2D views remains a fundamental challenge in computer vision. Conventional methods often decouple semantic understanding from reconstruction or necessitate costly per-scene optimization, thereby restricting their scalability and generalizability. In this paper, we introduce Uni3R, a novel feed-forward framework that jointly reconstructs a unified 3D scene representation enriched with open-vocabulary semantics, directly from unposed multi-view images. Our approach leverages a Cross-View Transformer to robustly integrate information across arbitrary multi-view inputs, which then regresses a set of 3D Gaussian primitives endowed with semantic feature fields. This unified representation facilitates high-fidelity novel view synthesis, open-vocabulary 3D semantic segmentation, and depth prediction, all within a single, feed-forward pass. Extensive experiments demonstrate that Uni3R establishes a new state-of-the-art across multiple benchmarks, including 25.07 PSNR on RE10K and 55.84 mIoU on ScanNet. Our work signifies a novel paradigm towards generalizable, unified 3D scene reconstruction and understanding. The code is available at https://github.com/HorizonRobotics/Uni3R.