# Canonical Space Representation for 4D Panoptic Segmentation of Articulated Objects

arXiv: https://arxiv.org/abs/2511.05356 · HTML: https://mlsanity.com/paper/2511.05356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 7 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [Is synthetic data from generative models ready for image recognition?](https://mlsanity.com/paper/2210.07574/) (2022)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Ditto: Building Digital Twins of Articulated Objects from Interaction](https://mlsanity.com/paper/2202.08227/) (2022)
- [PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects](https://mlsanity.com/paper/2308.07391/) (2023)
- [Synthetic Data for Deep Learning](https://mlsanity.com/paper/1909.11512/) (2019)
- [CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds](https://mlsanity.com/paper/2104.03437/) (2021)
- [AKB-48: A Real-World Articulated Object Knowledge Base](https://mlsanity.com/paper/2202.08432/) (2022)
- [Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects](https://mlsanity.com/paper/2404.01440/) (2024)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects](https://mlsanity.com/paper/2303.15782/) (2023)
- [Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmap](https://mlsanity.com/paper/2205.07002/) (2022)
- [Synthetic Datasets for Autonomous Driving: A Survey](https://mlsanity.com/paper/2304.12205/) (2023)
- [REACTO: Reconstructing Articulated Objects from a Single Video](https://mlsanity.com/paper/2404.11151/) (2024)
- [Towards Real-World Category-level Articulation Pose Estimation](https://mlsanity.com/paper/2105.03260/) (2021)
- [4D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregation](https://mlsanity.com/paper/2209.14858/) (2022)
- [Mask4Former: Mask Transformer for 4D Panoptic Segmentation](https://mlsanity.com/paper/2309.16133/) (2023)
- [4D Panoptic Segmentation as Invariant and Equivariant Field Prediction](https://mlsanity.com/paper/2303.15651/) (2023)
- [Semi-Weakly Supervised Object Kinematic Motion Prediction](https://mlsanity.com/paper/2303.17774/) (2023)
- [CAPT: Category-level Articulation Estimation from a Single Point Cloud Using Transformer](https://mlsanity.com/paper/2402.17360/) (2024)
- [MARS: Multimodal Active Robotic Sensing for Articulated Characterization](https://mlsanity.com/paper/2407.01191/) (2024)

## Abstract (arXiv)

Articulated object perception presents significant challenges in computer vision, particularly because most existing methods ignore temporal dynamics despite the inherently dynamic nature of such objects. The use of 4D temporal data has not been thoroughly explored in articulated object perception and remains unexamined for panoptic segmentation. The lack of a benchmark dataset further hurt this field. To this end, we introduce Artic4D as a new dataset derived from PartNet Mobility and augmented with synthetic sensor data, featuring 4D panoptic annotations and articulation parameters. Building on this dataset, we propose CanonSeg4D, a novel 4D panoptic segmentation framework. This approach explicitly estimates per-frame offsets mapping observed object parts to a learned canonical space, thereby enhancing part-level segmentation. The framework employs this canonical representation to achieve consistent alignment of object parts across sequential frames. Comprehensive experiments on Artic4D demonstrate that the proposed CanonSeg4D outperforms state of the art approaches in panoptic segmentation accuracy in more complex scenarios. These findings highlight the effectiveness of temporal modeling and canonical alignment in dynamic object understanding, and pave the way for future advances in 4D articulated object perception.