# NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Clouds

arXiv: https://arxiv.org/abs/2207.09978 · HTML: https://mlsanity.com/paper/2207.09978/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 55 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Frame Averaging for Invariant and Equivariant Network Design](https://mlsanity.com/paper/2110.03336/) (2021)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021)
- [Semantic Instance Segmentation with a Discriminative Loss Function](https://mlsanity.com/paper/1708.02551/) (2017)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [Associatively Segmenting Instances and Semantics in Point Clouds](https://mlsanity.com/paper/1902.09852/) (2019)

## Built on by (most cited first, 12 of 12)

- [OneFormer3D: One Transformer for Unified Point Cloud Segmentation](https://mlsanity.com/paper/2311.14405/) (2023)
- [Top-Down Beats Bottom-Up in 3D Instance Segmentation](https://mlsanity.com/paper/2302.02871/) (2023)
- [A Unified Framework for 3D Scene Understanding](https://mlsanity.com/paper/2407.03263/) (2024)
- [Structured Epipolar Matcher for Local Feature Matching](https://mlsanity.com/paper/2303.16646/) (2023)
- [MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation](https://mlsanity.com/paper/2411.01781/) (2024)
- [CompetitorFormer: Competitor Transformer for 3D Instance Segmentation](https://mlsanity.com/paper/2411.14179/) (2024)
- [All in One: Visual-Description-Guided Unified Point Cloud Segmentation](https://mlsanity.com/paper/2507.05211/) (2025)
- [ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method](https://mlsanity.com/paper/2410.02352/) (2024)
- [Joint prototype and coefficient prediction for 3D instance segmentation](https://mlsanity.com/paper/2407.06958/) (2024)
- [Towards scalable organ level 3D plant segmentation: Bridging the data algorithm computing gap](https://mlsanity.com/paper/2509.06329/) (2025)
- [Weakly Supervised Point Cloud Segmentation via Conservative Propagation of Scene-level Labels](https://mlsanity.com/paper/2312.06799/) (2023)
- [Instance Segmentation of Reinforced Concrete Bridges with Synthetic Point Clouds](https://mlsanity.com/paper/2409.16381/) (2024)

## Abstract (arXiv)

We introduce a method for instance proposal generation for 3D point clouds. Existing techniques typically directly regress proposals in a single feed-forward step, leading to inaccurate estimation. We show that this serves as a critical bottleneck, and propose a method based on iterative bilateral filtering with learned kernels. Following the spirit of bilateral filtering, we consider both the deep feature embeddings of each point, as well as their locations in the 3D space. We show via synthetic experiments that our method brings drastic improvements when generating instance proposals for a given point of interest. We further validate our method on the challenging ScanNet benchmark, achieving the best instance segmentation performance amongst the sub-category of top-down methods.