# Weakly Supervised 3D Object Detection via Multi-Level Visual Guidance

arXiv: https://arxiv.org/abs/2312.07530 · HTML: https://mlsanity.com/paper/2312.07530/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.11301/) (2023)
- [Unifying Voxel-based Representation with Transformer for 3D Object Detection](https://mlsanity.com/paper/2206.00630/) (2022)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformer](https://mlsanity.com/paper/2203.10981/) (2022)
- [CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2012.03015/) (2020)
- [Weakly Supervised 3D Object Detection from Lidar Point Cloud](https://mlsanity.com/paper/2007.11901/) (2020)
- [Weakly Supervised 3D Object Detection from Point Clouds](https://mlsanity.com/paper/2007.13970/) (2020)
- [M3DSSD: Monocular 3D Single Stage Object Detector](https://mlsanity.com/paper/2103.13164/) (2021)
- [FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection](https://mlsanity.com/paper/2105.07647/) (2021)
- [PG-RCNN: Semantic Surface Point Generation for 3D Object Detection](https://mlsanity.com/paper/2307.12637/) (2023)
- [Multimodal Transformer for Automatic 3D Annotation and Object Detection](https://mlsanity.com/paper/2207.09805/) (2022)
- [MAP-Gen: An Automated 3D-Box Annotation Flow with Multimodal Attention Point Generator](https://mlsanity.com/paper/2203.15700/) (2022)
- [Multi-Stream Attention Learning for Monocular Vehicle Velocity and Inter-Vehicle Distance Estimation](https://mlsanity.com/paper/2110.11608/) (2021)

## Built on by (most cited first, 9 of 9)

- [Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments](https://mlsanity.com/paper/2403.13556/) (2024)
- [PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection](https://mlsanity.com/paper/2312.08371/) (2023)
- [MonoSOWA: Scalable monocular 3D Object detector Without human Annotations](https://mlsanity.com/paper/2501.09481/) (2025)
- [CA-W3D: Leveraging Context-Aware Knowledge for Weakly Supervised Monocular 3D Detection](https://mlsanity.com/paper/2503.04154/) (2025)
- [PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection](https://mlsanity.com/paper/2507.02393/) (2025)
- [PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents](https://mlsanity.com/paper/2605.10341/) (2026)
- [MVAT: Multi-View Aware Teacher for Weakly Supervised 3D Object Detection](https://mlsanity.com/paper/2509.07507/) (2025)
- [Auto-Vocabulary 3D Object Detection](https://mlsanity.com/paper/2512.16077/) (2025)
- [End-to-End LiDAR optimization for 3D point cloud registration](https://mlsanity.com/paper/2602.10492/) (2026)

## Abstract (arXiv)

Weakly supervised 3D object detection aims to learn a 3D detector with lower annotation cost, e.g., 2D labels. Unlike prior work which still relies on few accurate 3D annotations, we propose a framework to study how to leverage constraints between 2D and 3D domains without requiring any 3D labels. Specifically, we employ visual data from three perspectives to establish connections between 2D and 3D domains. First, we design a feature-level constraint to align LiDAR and image features based on object-aware regions. Second, the output-level constraint is developed to enforce the overlap between 2D and projected 3D box estimations. Finally, the training-level constraint is utilized by producing accurate and consistent 3D pseudo-labels that align with the visual data. We conduct extensive experiments on the KITTI dataset to validate the effectiveness of the proposed three constraints. Without using any 3D labels, our method achieves favorable performance against state-of-the-art approaches and is competitive with the method that uses 500-frame 3D annotations. Code will be made publicly available at https://github.com/kuanchihhuang/VG-W3D.