# Co-Win: Joint Object Detection and Instance Segmentation in LiDAR Point Clouds via Collaborative Window Processing

arXiv: https://arxiv.org/abs/2507.19691 · HTML: https://mlsanity.com/paper/2507.19691/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/1910.06528/) (2019)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [3D Object Detection for Autonomous Driving: A Survey](https://mlsanity.com/paper/2106.10823/) (2021)
- [PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things](https://mlsanity.com/paper/1903.01177/) (2019)
- [LiDAR-based Panoptic Segmentation via Dynamic Shifting Network](https://mlsanity.com/paper/2011.11964/) (2020)
- [Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction](https://mlsanity.com/paper/2304.00967/) (2023)
- [Fully Sparse Fusion for 3D Object Detection](https://mlsanity.com/paper/2304.12310/) (2023)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Mask2Map: Vectorized HD Map Construction Using Bird's Eye View Segmentation Masks](https://mlsanity.com/paper/2407.13517/) (2024)
- [PolarStream: Streaming Lidar Object Detection and Segmentation with Polar Pillars](https://mlsanity.com/paper/2106.07545/) (2021)
- [MaskBEV: Towards A Unified Framework for BEV Detection and Map Segmentation](https://mlsanity.com/paper/2408.09122/) (2024)
- [MaskBEV: Joint Object Detection and Footprint Completion for Bird's-eye View 3D Point Clouds](https://mlsanity.com/paper/2307.01864/) (2023)

## Abstract (arXiv)

Accurate perception and scene understanding in complex urban environments is a critical challenge for ensuring safe and efficient autonomous navigation. In this paper, we present Co-Win, a novel bird's eye view (BEV) perception framework that integrates point cloud encoding with efficient parallel window-based feature extraction to address the multi-modality inherent in environmental understanding. Our method employs a hierarchical architecture comprising a specialized encoder, a window-based backbone, and a query-based decoder head to effectively capture diverse spatial features and object relationships. Unlike prior approaches that treat perception as a simple regression task, our framework incorporates a variational approach with mask-based instance segmentation, enabling fine-grained scene decomposition and understanding. The Co-Win architecture processes point cloud data through progressive feature extraction stages, ensuring that predicted masks are both data-consistent and contextually relevant. Furthermore, our method produces interpretable and diverse instance predictions, enabling enhanced downstream decision-making and planning in autonomous driving systems.