# GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision

arXiv: https://arxiv.org/abs/2007.13124 · HTML: https://mlsanity.com/paper/2007.13124/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2020.

## Built on by (most cited first, 28 of 28)

- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association](https://mlsanity.com/paper/2103.02440/) (2021)
- [Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image](https://mlsanity.com/paper/2109.06161/) (2021)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Fine-Grained Object Classification via Self-Supervised Pose Alignment](https://mlsanity.com/paper/2203.15987/) (2022)
- [Certifiably Optimal Outlier-Robust Geometric Perception: Semidefinite Relaxations and Scalable Global Optimization](https://mlsanity.com/paper/2109.03349/) (2021)
- [ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation](https://mlsanity.com/paper/2111.10524/) (2021)
- [Optimal Pose and Shape Estimation for Category-level 3D Object Perception](https://mlsanity.com/paper/2104.08383/) (2021)
- [Keypoint Communities](https://mlsanity.com/paper/2110.00988/) (2021)
- [Semantic keypoint-based pose estimation from single RGB frames](https://mlsanity.com/paper/2204.05864/) (2022)
- [Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints](https://mlsanity.com/paper/2206.12498/) (2022)
- [Toward Planet-Wide Traffic Camera Calibration](https://mlsanity.com/paper/2311.04243/) (2023)
- [HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language Models](https://mlsanity.com/paper/2409.10419/) (2024)
- [Dynamical Pose Estimation](https://mlsanity.com/paper/2103.06182/) (2021)
- [Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild](https://mlsanity.com/paper/2111.12728/) (2021)
- [Occlusion-Aware Instance Segmentation via BiLayer Network Architectures](https://mlsanity.com/paper/2208.04438/) (2022)
- [RESSCAL3D: Resolution Scalable 3D Semantic Segmentation of Point Clouds](https://mlsanity.com/paper/2404.06863/) (2024)
- [Stereo Neural Vernier Caliper](https://mlsanity.com/paper/2203.11018/) (2022)
- [WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusion](https://mlsanity.com/paper/2403.19022/) (2024)
- [GrabS: Generative Embodied Agent for 3D Object Segmentation without Scene Supervision](https://mlsanity.com/paper/2504.11754/) (2025)
- [SenseShift6D: Multimodal RGB-D Benchmarking for Robust 6D Pose Estimation across Environment and Sensor Variations](https://mlsanity.com/paper/2507.05751/) (2025)
- [Category-Level Object Shape and Pose Estimation in Less Than a Millisecond](https://mlsanity.com/paper/2509.18979/) (2025)
- [Joint stereo 3D object detection and implicit surface reconstruction](https://mlsanity.com/paper/2111.12924/) (2021)
- [DeepKalPose: An Enhanced Deep-Learning Kalman Filter for Temporally Consistent Monocular Vehicle Pose Estimation](https://mlsanity.com/paper/2404.16558/) (2024)
- [Unposed-to-3D: Learning Simulation-Ready Vehicles from Real-World Images](https://mlsanity.com/paper/2604.19257/) (2026)
- [Canonical mapping as a general-purpose object descriptor for robotic manipulation](https://mlsanity.com/paper/2303.01331/) (2023)
- [Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction](https://mlsanity.com/paper/2501.14896/) (2025)

## Abstract (arXiv)

We present a novel end-to-end framework named as GSNet (Geometric and Scene-aware Network), which jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view. GSNet utilizes a unique four-way feature extraction and fusion scheme and directly regresses 6DoF poses and shapes in a single forward pass. Extensive experiments show that our diverse feature extraction and fusion scheme can greatly improve model performance. Based on a divide-and-conquer 3D shape representation strategy, GSNet reconstructs 3D vehicle shape with great detail (1352 vertices and 2700 faces). This dense mesh representation further leads us to consider geometrical consistency and scene context, and inspires a new multi-objective loss function to regularize network training, which in turn improves the accuracy of 6D pose estimation and validates the merit of jointly performing both tasks. We evaluate GSNet on the largest multi-task ApolloCar3D benchmark and achieve state-of-the-art performance both quantitatively and qualitatively. Project page is available at https://lkeab.github.io/gsnet/.