# ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description arXiv: https://arxiv.org/abs/2108.00355 ยท HTML: https://mlsanity.com/paper/2108.00355/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 51 references that are in the map. - First posted on arXiv 1 Aug 2021. ## Builds on (most cited first, 30 of 30) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015) - [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019) - [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018) - [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018) - [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016) - [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018) - [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019) - [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019) - [ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals](https://mlsanity.com/paper/1905.02082/) (2019) - [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018) - [CubeSLAM: Monocular 3D Object SLAM](https://mlsanity.com/paper/1806.00557/) (2018) - [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018) - [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018) - [DualSDF: Semantic Shape Manipulation using a Two-Level Representation](https://mlsanity.com/paper/2004.02869/) (2020) - [NodeSLAM: Neural Object Descriptors for Multi-View Shape Reconstruction](https://mlsanity.com/paper/2004.04485/) (2020) - [FroDO: From Detections to 3D Objects](https://mlsanity.com/paper/2005.05125/) (2020) - [End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans](https://mlsanity.com/paper/1906.04201/) (2019) - [Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videos](https://mlsanity.com/paper/2012.04641/) (2020) - [Meaningful Maps With Object-Oriented Semantic Mapping](https://mlsanity.com/paper/1609.07849/) (2016) - [Learning to Group: A Bottom-Up Framework for 3D Part Discovery in Unseen Categories](https://mlsanity.com/paper/2002.06478/) (2020) - [Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoning](https://mlsanity.com/paper/1807.05933/) (2018) - [EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association](https://mlsanity.com/paper/2004.12730/) (2020) - [Real-time Scalable Dense Surfel Mapping](https://mlsanity.com/paper/1909.04250/) (2019) - [MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videos](https://mlsanity.com/paper/2012.05360/) (2020) - [CAD-Deform: Deformable Fitting of CAD Models to 3D Scans](https://mlsanity.com/paper/2007.11965/) (2020) - [Beyond Holistic Object Recognition: Enriching Image Understanding with Part States](https://mlsanity.com/paper/1612.07310/) (2016) - [Deep-SLAM++: Object-level RGBD SLAM based on class-specific deep shape priors](https://mlsanity.com/paper/1907.09691/) (2019) - [SLAM with Objects using a Nonparametric Pose Graph](https://mlsanity.com/paper/1704.05959/) (2017) - [Extending DeepSDF for automatic 3D shape retrieval and similarity transform estimation](https://mlsanity.com/paper/2004.09048/) (2020) ## Built on by (most cited first, 10 of 10) - [6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model](https://mlsanity.com/paper/2407.15484/) (2024) - [Generative Category-Level Shape and Pose Estimation with Semantic Primitives](https://mlsanity.com/paper/2210.01112/) (2022) - [RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformers](https://mlsanity.com/paper/2203.13296/) (2022) - [Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments](https://mlsanity.com/paper/2312.09138/) (2023) - [O2-Recon: Completing 3D Reconstruction of Occluded Objects in the Scene with a Pre-trained 2D Diffusion Model](https://mlsanity.com/paper/2308.09591/) (2023) - [TransPoser: Transformer as an Optimizer for Joint Object Shape and Pose Estimation](https://mlsanity.com/paper/2303.13477/) (2023) - [ShapeICP: Iterative Category-level Object Pose and Shape Estimation from Depth](https://mlsanity.com/paper/2408.13147/) (2024) - [Box Pose and Shape Estimation and Domain Adaptation for Large-Scale Warehouse Automation](https://mlsanity.com/paper/2507.00984/) (2025) - [Taylor3DNet: Fast 3D Shape Inference With Landmark Points Based Taylor Series](https://mlsanity.com/paper/2201.06845/) (2022) - [Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation](https://mlsanity.com/paper/2607.09455/) (2026) ## Abstract (arXiv) Autonomous systems need to understand the semantics and geometry of their surroundings in order to comprehend and safely execute object-level task specifications. This paper proposes an expressive yet compact model for joint object pose and shape optimization, and an associated optimization algorithm to infer an object-level map from multi-view RGB-D camera observations. The model is expressive because it captures the identities, positions, orientations, and shapes of objects in the environment. It is compact because it relies on a low-dimensional latent representation of implicit object shape, allowing onboard storage of large multi-category object maps. Different from other works that rely on a single object representation format, our approach has a bi-level object model that captures both the coarse level scale as well as the fine level shape details. Our approach is evaluated on the large-scale real-world ScanNet dataset and compared against state-of-the-art methods.