# Segment Any 4D Gaussians

arXiv: https://arxiv.org/abs/2407.04504 · HTML: https://mlsanity.com/paper/2407.04504/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 77 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 30 of 68)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Gaussian Grouping: Segment and Edit Anything in 3D Scenes](https://mlsanity.com/paper/2312.00732/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)

## Built on by (most cited first, 20 of 20)

- [4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models](https://mlsanity.com/paper/2503.10437/) (2025)
- [4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives](https://mlsanity.com/paper/2412.20720/) (2024)
- [Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fields](https://mlsanity.com/paper/2503.20776/) (2025)
- [Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization](https://mlsanity.com/paper/2507.23569/) (2025)
- [TRASE: Tracking-free 4D Segmentation and Editing](https://mlsanity.com/paper/2411.19290/) (2024)
- [Consistent Instance Field for Dynamic Scene Understanding](https://mlsanity.com/paper/2512.14126/) (2025)
- [Semantic Consistent Language Gaussian Splatting for Point-Level Open-vocabulary Querying](https://mlsanity.com/paper/2503.21767/) (2025)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding](https://mlsanity.com/paper/2512.03601/) (2025)
- [4D Synchronized Fields: Motion-Language Gaussian Splatting for Temporal Scene Understanding](https://mlsanity.com/paper/2603.14301/) (2026)
- [Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction](https://mlsanity.com/paper/2605.31595/) (2026)
- [ROODI: Reconstructing Occluded Objects with Denoising Inpainters](https://mlsanity.com/paper/2503.10256/) (2025)
- [TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation](https://mlsanity.com/paper/2602.08540/) (2026)
- [PanopticQuery: Unified Query-Time Reasoning for 4D Scenes](https://mlsanity.com/paper/2604.05638/) (2026)
- [Leveraging 2D Priors and SDF Guidance for Dynamic Urban Scene Rendering](https://mlsanity.com/paper/2510.13381/) (2025)
- [Director: Instance-aware Gaussian Splatting for Dynamic Scene Modeling and Understanding](https://mlsanity.com/paper/2604.01678/) (2026)
- [Accelerated Spatio-Temporal Gaussian Rendering via Kinematic and Semantic Priors](https://mlsanity.com/paper/2605.25909/) (2026)
- [Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocation](https://mlsanity.com/paper/2606.22197/) (2026)
- [Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer](https://mlsanity.com/paper/2608.20659/) (2026)
- [S4VY: Segment Anything in Feed-Forward 4D Visual Geometry](https://mlsanity.com/paper/2609.36875/) (2026)

## Abstract (arXiv)

Modeling, understanding, and reconstructing the real world are crucial in XR/VR. Recently, 3D Gaussian Splatting (3D-GS) methods have shown remarkable success in modeling and understanding 3D scenes. Similarly, various 4D representations have demonstrated the ability to capture the dynamics of the 4D world. However, there is a dearth of research focusing on segmentation within 4D representations. In this paper, we propose Segment Any 4D Gaussians (SA4D), one of the first frameworks to segment anything in the 4D digital world based on 4D Gaussians. In SA4D, an efficient temporal identity feature field is introduced to handle Gaussian drifting, with the potential to learn precise identity features from noisy and sparse input. Additionally, a 4D segmentation refinement process is proposed to remove artifacts. Our SA4D achieves precise, high-quality segmentation within seconds in 4D Gaussians and shows the ability to remove, recolor, compose, and render high-quality anything masks. More demos are available at: https://jsxzs.github.io/sa4d/.