# 3D-Aware Object Localization using Gaussian Implicit Occupancy Function

arXiv: https://arxiv.org/abs/2303.02058 · HTML: https://mlsanity.com/paper/2303.02058/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 3 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Numerical Coordinate Regression with Convolutional Neural Networks](https://mlsanity.com/paper/1801.07372/) (2018)
- [Satellite Pose Estimation Challenge: Dataset, Competition Design and Results](https://mlsanity.com/paper/1911.02050/) (2019)
- [SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap](https://mlsanity.com/paper/2110.03101/) (2021)
- [Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering](https://mlsanity.com/paper/1907.04298/) (2019)
- [Ellipse R-CNN: Learning to Infer Elliptical Object from Clustering and Occlusion](https://mlsanity.com/paper/2001.11584/) (2020)
- [LSPnet: A 2D Localization-oriented Spacecraft Pose Estimation Neural Network](https://mlsanity.com/paper/2104.09248/) (2021)
- [Ellipse Detection and Localization with Applications to Knots in Sawn Lumber Images](https://mlsanity.com/paper/2011.04844/) (2020)
- [Detecting Lesion Bounding Ellipses With Gaussian Proposal Networks](https://mlsanity.com/paper/1902.09658/) (2019)
- [3D-Aware Ellipse Prediction for Object-Based Camera Pose Estimation](https://mlsanity.com/paper/2105.11494/) (2021)
- [Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence](https://mlsanity.com/paper/2208.12513/) (2022)
- [Object-Based Visual Camera Pose Estimation From Ellipsoidal Model and 3D-Aware Ellipse Prediction](https://mlsanity.com/paper/2203.04613/) (2022)
- [Level Set-Based Camera Pose Estimation From Multiple 2D/3D Ellipse-Ellipsoid Correspondences](https://mlsanity.com/paper/2207.07953/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Survey on Deep Learning-Based Monocular Spacecraft Pose Estimation: Current State, Limitations and Prospects](https://mlsanity.com/paper/2305.07348/) (2023)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)

## Abstract (arXiv)

To automatically localize a target object in an image is crucial for many computer vision applications. To represent the 2D object, ellipse labels have recently been identified as a promising alternative to axis-aligned bounding boxes. This paper further considers 3D-aware ellipse labels, textiti.e., ellipses which are projections of a 3D ellipsoidal approximation of the object, for 2D target localization. Indeed, projected ellipses carry more geometric information about the object geometry and pose (3D awareness) than traditional 3D-agnostic bounding box labels. Moreover, such a generic 3D ellipsoidal model allows for approximating known to coarsely known targets. We then propose to have a new look at ellipse regression and replace the discontinuous geometric ellipse parameters with the parameters of an implicit Gaussian distribution encoding object occupancy in the image. The models are trained to regress the values of this bivariate Gaussian distribution over the image pixels using a statistical loss function. We introduce a novel non-trainable differentiable layer, E-DSNT, to extract the distribution parameters. Also, we describe how to readily generate consistent 3D-aware Gaussian occupancy parameters using only coarse dimensions of the target and relative pose labels. We extend three existing spacecraft pose estimation datasets with 3D-aware Gaussian occupancy labels to validate our hypothesis. Labels and source code are publicly accessible here: https://cvi2.uni.lu/3d-aware-obj-loc/.