mlsanityData to 7 Oct 2026

OneFormer3D: One Transformer for Unified Point Cloud Segmentation

arXiv 2311.14405 ↗First posted 24 Nov 2023cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

Semantic, instance, and panoptic segmentation of 3D point clouds have been addressed using task-specific models of distinct design. Thereby, the similarity of all segmentation tasks and the implicit relationship between them have not been utilized effectively. This paper presents a unified, simple, and effective model addressing all these tasks jointly. The model, named OneFormer3D, performs instance and semantic segmentation consistently, using a group of learnable kernels, where each kernel is responsible for generating a mask for either an instance or a semantic category. These kernels are trained with a transformer-based decoder with unified instance and semantic queries passed as an input. Such a design enables training a model end-to-end in a single run, so that it achieves top performance on all three segmentation tasks simultaneously. Specifically, our OneFormer3D ranks 1st and sets a new state-of-the-art (+2.1 mAP50) in the ScanNet test leaderboard. We also demonstrate the state-of-the-art results in semantic, instance, and panoptic segmentation of ScanNet (+21 PQ), ScanNet200 (+3.8 mAP50), and S3DIS (+0.8 mIoU) datasets.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 44 of its 52 references that are in the map. First posted on arXiv 24 Nov 2023.

Builds on

44 of 52 references.

References · 44

  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2020Point Transformercited by 1,462
  7. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  8. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  10. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  11. 2018Panoptic Segmentationcited by 796
  12. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
32 more
  1. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  2. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
  3. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  4. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  5. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  6. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  7. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  8. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
  9. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  10. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 243
  11. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  12. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  13. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  14. 2021K-Net: Towards Unified Image Segmentationcited by 211
  15. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  16. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  17. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
  18. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 145
  19. 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 126
  20. 2022Superpoint Transformer for 3D Scene Instance Segmentationcited by 115
  21. 2021Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networkscited by 98
  22. 2019PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Thingscited by 97
  23. 2021FCAF3D: Fully Convolutional Anchor-Free 3D Object Detectioncited by 97
  24. 2020DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolutioncited by 74
  25. 2022CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Cloudscited by 58
  26. 2023ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolutioncited by 52
  27. 2022Meta Architecture for Point Cloud Analysiscited by 39
  28. 20223D Instances as 1D Kernelscited by 36
  29. 2023TR3D: Towards Real-Time Indoor 3D Object Detectioncited by 32
  30. 2023Top-Down Beats Bottom-Up in 3D Instance Segmentationcited by 31
  31. 2022Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarizationcited by 26
  32. 2022NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Cloudscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 20; 2025: 36; 2026: 38.

95 papers.

Sort:

By area:

  1. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  2. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 108
  3. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 67
  4. 2024EmbodiedSAM: Online Segment Any 3D Thing in Real Timecited by 40
  5. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 34
  6. 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
  7. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Modelcited by 27
  8. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 26
  9. 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
  10. 2025OpenSplat3D: Open-Vocabulary 3D Instance Segmentation using Gaussian Splattingcited by 19
  11. 2025Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Modelcited by 17
  12. 2024A Unified Framework for 3D Scene Understandingcited by 15
38 more
  1. 2025SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignmentcited by 15
  2. 2024SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentationcited by 12
  3. 2025ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Cloudscited by 12
  4. 2024Grounding 3D Scene Affordance From Egocentric Interactionscited by 11
  5. 2024Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphscited by 10
  6. 2025LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Cloudscited by 8
  7. 2025Details Matter for Indoor Open-vocabulary 3D Instance Segmentationcited by 8
  8. 2025Registration is a Powerful Rotation-Invariance Learner for 3D Anomaly Detectioncited by 7
  9. 2026FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videoscited by 7
  10. 2024Swin3D++: Effective Multi-Source Pretraining for 3D Indoor Scene Understandingcited by 6
  11. 2025UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splattingcited by 6
  12. 2025Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scalecited by 6
  13. 2025MoE3D: Mixture of Experts meets Multi-Modal 3D Understandingcited by 6
  14. 2024MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processingcited by 5
  15. 2025SAS: Segment Any 3D Scene with Integrated 2D Priorscited by 5
  16. 2024CompetitorFormer: Competitor Transformer for 3D Instance Segmentationcited by 4
  17. 2024CADSpotting: Robust Panoptic Symbol Spotting on Large-Scale CAD Drawingscited by 4
  18. 2025Sketchy Bounding-box Supervision for 3D Instance Segmentationcited by 4
  19. 2025All in One: Visual-Description-Guided Unified Point Cloud Segmentationcited by 4
  20. 2025OVSeg3R: Learn Open-vocabulary Instance Segmentation from 2D via 3D Reconstructioncited by 4
  21. 2025Online Segment Any 3D Thing as Instance Trackingcited by 4
  22. 2026Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understandingcited by 4
  23. 2024AutoInst: Automatic Instance-Based Segmentation of LiDAR 3D Scanscited by 3
  24. 2025GrabS: Generative Embodied Agent for 3D Object Segmentation without Scene Supervisioncited by 3
  25. 2025Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawingscited by 3
  26. 2025PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculumcited by 3
  27. 2025Differentiable High-Performance Ray Tracing-Based Simulation of Radio Propagation with Point Cloudscited by 3
  28. 2025OpenTrack3D: Towards Accurate and Generalizable Open-Vocabulary 3D Instance Segmentationcited by 3
  29. 20263AM: 3egment Anything with Geometric Consistency in Videoscited by 3
  30. 2026LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentationcited by 3
  31. 2026ForestMamba: Sparse Mamba with Geometry-guided Queries for 3D Forest Point Cloud Segmentationcited by 3
  32. 2025Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branchescited by 2
  33. 2025Easy3D: A Simple Yet Effective Method for 3D Interactive Segmentationcited by 2
  34. 2025Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Modelcited by 2
  35. 2025Benchmarking individual tree segmentation using multispectral airborne laser scanning data: the FGI-EMIT datasetcited by 2
  36. 2025Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physicscited by 2
  37. 2026End-to-end event reconstruction for precision physics at future colliderscited by 2
  38. 2026Segment and Select: Vision-Language Segmentation in 3D Scenarioscited by 2

50 most cited of 95 All 95 (JSON)