UniFunc3D: Unified Active Spatial-Temporal Grounding for 3D Affordance Segmentation
Not on the map. Why?
Abstract (arXiv)
Affordance segmentation in 3D scenes requires an agent to ground implicit natural-language instructions into precise masks of fine-grained interactive elements. Existing training-free methods typically rely on fragmented pipelines, which introduce visual blindness during task parsing and limit accuracy through single-scale spatial and temporal processing. We present UniFunc3D, a unified and training-free framework that treats the multimodal large language model as an active observer. By utilizing a unified MLLM backbone, UniFunc3D performs joint semantic-temporal-spatial reasoning to ground task decomposition in direct visual evidence. Our approach introduces active spatial-temporal grounding with a coarse-to-fine strategy. This allows the model to select correct video frames adaptively and focus on high-detail interactive parts while preserving the global context necessary for disambiguation. On SceneFun3D, our UniFunc3D achieves state-of-the-art performance, surpassing prior training-free methods by a large margin with a relative 59.9% mIoU improvement, and even outperforming training-based methods without any task-specific training. Code is available on our project page: urlhttps://jiaying.link/unifunc3d.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 42 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
37 of 42 references.
References · 37
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Visual Instruction Tuningcited by 7,792
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
- 2023Segment Everything Everywhere All at Oncecited by 546
- 2023LERF: Language Embedded Radiance Fieldscited by 538
- 2023LangSplat: 3D Language Gaussian Splattingcited by 410
25 more
- 2023Scaling Open-Vocabulary Object Detectioncited by 335
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
- 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 123
- 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
- 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
- 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
- 2024OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Viewscited by 68
- 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
- 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
- 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
- 2023One-Shot Open Affordance Learning with Foundation Modelscited by 45
- 2024RobustSAM: Segment Anything Robustly on Degraded Imagescited by 36
- 20253D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worldscited by 29
- 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Modelscited by 28
- 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 22
- 2024Functionality understanding and segmentation in 3D scenescited by 17
- 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 17
- 2025AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Modelscited by 15
- 2024Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Descriptioncited by 9
- 2025KeySG: Hierarchical Keyframe-Based 3D Scene Graphscited by 8
- 2024Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understandingcited by 6
- 2025Task-Aware 3D Affordance Segmentation via 2D Guidance and Geometric Refinementcited by 6
- 2024Zero-Shot Dual-Path Integration Framework for Open-Vocabulary 3D Instance Segmentationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
- 2024Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models
- 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
- 2026Grounded 3D-Aware Spatial Vision-Language Modeling
- 2026SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding
- 2025Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotations
- 2025Action-guided generation of 3D functionality segmentation data
- 2025SORT3D: Spatial Object-centric Reasoning Toolbox for Zero-Shot 3D Grounding Using Large Language Models