mlsanityData to 7 Oct 2026

GARField: Group Anything with Radiance Fields

arXiv 2401.09419 ↗First posted 17 Jan 2024cs.CV cs.GRcited by 108

Not on the map. Why?

Abstract (arXiv)

Grouping is inherently ambiguous due to the multiple levels of granularity in which one can decompose a scene -- should the wheels of an excavator be considered separate or part of the whole? We present Group Anything with Radiance Fields (GARField), an approach for decomposing 3D scenes into a hierarchy of semantically meaningful groups from posed image inputs. To do this we embrace group ambiguity through physical scale: by optimizing a scale-conditioned 3D affinity feature field, a point in the world can belong to different groups of different sizes. We optimize this field from a set of 2D masks provided by Segment Anything (SAM) in a way that respects coarse-to-fine hierarchy, using scale to consistently fuse conflicting masks from different viewpoints. From this field we can derive a hierarchy of possible groupings via automatic tree construction or user interaction. We evaluate GARField on a variety of in-the-wild scenes and find it effectively extracts groups at many levels: clusters of objects, objects, and various subparts. GARField inherently represents multi-view consistent groupings and produces higher fidelity groups than the input SAM masks. GARField's hierarchical grouping could have exciting downstream applications such as 3D asset extraction or dynamic scene understanding. See the project website at https://www.garfield.studio/

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

26 of 38 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2017Mask R-CNNcited by 7,259
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2018Panoptic Segmentationcited by 796
  8. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  9. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  10. 2015Holistically-Nested Edge Detectioncited by 590
  11. 2023LERF: Language Embedded Radiance Fieldscited by 538
  12. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
14 more
  1. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  2. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  3. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  4. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  5. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  6. 2015Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generationcited by 90
  7. 2020Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Imagecited by 72
  8. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  9. 2023Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusioncited by 62
  10. 2022Neural Volumetric Object Selectioncited by 61
  11. 2019StructureNet: Hierarchical Graph Networks for 3D Shape Generationcited by 59
  12. 2022Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformerscited by 39
  13. 2023Instance Neural Radiance Fieldcited by 39
  14. 2023Interactive Segment Anything NeRF with Feature Imitationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 33; 2025: 48; 2026: 27.

108 papers.

Sort:

By area:

  1. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 95
  2. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 91
  3. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
  4. 2024Click-Gaussian: Interactive Segmentation to Any 3D Gaussianscited by 58
  5. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 46
  6. 2024Gaga: Group Any Gaussians via 3D-aware Memory Bankcited by 40
  7. 2024WaterSplatting: Fast Underwater 3D Scene Reconstruction Using Gaussian Splattingcited by 39
  8. 2024LUDVIG: Learning-Free Uplifting of 2D Visual Features to Gaussian Splatting Scenescited by 36
  9. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  10. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  11. 2025AutoPartGen: Autogressive 3D Part Generation and Discoverycited by 33
  12. 2024Search3D: Hierarchical Open-Vocabulary 3D Segmentationcited by 31
38 more
  1. 2024Find Any Part in 3Dcited by 30
  2. 2024SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussianscited by 30
  3. 2025Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splattingcited by 29
  4. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  5. 2024NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methodscited by 24
  6. 2025Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentationcited by 23
  7. 2024Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Renderingcited by 19
  8. 2025COB-GS: Clear Object Boundaries in 3DGS Segmentation Based on Boundary-Adaptive Gaussian Splittingcited by 19
  9. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 19
  10. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  11. 2024GAGS: Granularity-Aware Feature Distillation for Language Gaussian Splattingcited by 18
  12. 2024PLGS: Robust Panoptic Lifting with 3D Gaussian Splattingcited by 17
  13. 2024LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understandingcited by 17
  14. 2024RelationField: Relate Anything in Radiance Fieldscited by 16
  15. 20244D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitivescited by 15
  16. 2025SemanticSplat: Feed-Forward 3D Scene Understanding with Language-Aware Gaussian Fieldscited by 13
  17. 2025Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracingcited by 13
  18. 20244-LEGS: 4D Language Embedded Gaussian Splattingcited by 12
  19. 2025iSegMan: Interactive Segment-and-Manipulate 3D Gaussianscited by 11
  20. 2025InstaScene: Towards Complete 3D Instance Decomposition and Reconstruction from Cluttered Scenescited by 11
  21. 2024GradiSeg: Gradient-Guided Gaussian Segmentation with Enhanced 3D Boundary Precisioncited by 10
  22. 2024SAMa: Material-aware 3D Selection and Segmentationcited by 9
  23. 2024View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fieldscited by 8
  24. 2024Self-supervised Learning of Hybrid Part-aware 3D Representations of 2D Gaussians and Superquadricscited by 8
  25. 2024TRASE: Tracking-free 4D Segmentation and Editingcited by 8
  26. 2025Predict-Optimize-Distill: A Self-Improving Cycle for 4D Object Understandingcited by 8
  27. 2025COS3D: Collaborative Open-Vocabulary 3D Segmentationcited by 8
  28. 2024LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Controlcited by 6
  29. 2024OpenSU3D: Open World 3D Scene Understanding using Foundation Modelscited by 6
  30. 2025Pointmap Association and Piecewise-Plane Constraint for Consistent and Compact 3D Gaussian Segmentation Fieldcited by 6
  31. 2025Multimodal Spatial Language Maps for Robot Navigation and Manipulationcited by 6
  32. 2025A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Groundingcited by 6
  33. 2025GALA: Guided Attention with Language Alignment for Open Vocabulary Gaussian Splattingcited by 6
  34. 2025REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splattingcited by 6
  35. 2025OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusioncited by 6
  36. 20243D-Aware Instance Segmentation and Tracking in Egocentric Videoscited by 5
  37. 2025Leverage Cross-Attention for End-to-End Open-Vocabulary Panoptic Reconstructioncited by 5
  38. 2025OpenHype: Hyperbolic Embeddings for Hierarchical Open-Vocabulary Radiance Fieldscited by 5

50 most cited of 108 All 108 (JSON)