MoCA3D: Monocular 3D Bounding Box Prediction in the Image Plane
Not on the map. Why?
Abstract (arXiv)
Monocular 3D object understanding has largely been cast as a 2D RoI-to-3D box lifting problem. However, emerging downstream applications require image-plane geometry (e.g., projected 3D box corners) which cannot be easily obtained without known intrinsics, a problem for object detection in the wild. We introduce MoCA3D, a Monocular, Class-Agnostic 3D model that predicts projected 3D bounding box corners and per-corner depths without requiring camera intrinsics at inference time. MoCA3D formulates pixel-space localization and depth assignment as dense prediction via corner heatmaps and depth maps. To evaluate image-plane geometric fidelity, we propose Pixel-Aligned Geometry (PAG), which directly measures image-plane corner and depth consistency. Extensive experiments demonstrate that MoCA3D achieves state-of-the-art performance, improving image-plane corner PAG by 22.8% while remaining comparable on 3D IoU, using up to 57 times fewer trainable parameters. Finally, we apply MoCA3D to downstream tasks which were previously impractical under unknown intrinsics, highlighting its utility beyond standard baseline models.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 46 of its 59 references that are in the map. First posted on arXiv 20 Mar 2026.
Builds on
46 of 59 references.
References · 46
- 2017Attention Is All You Needcited by 60,106
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
- 2020End-to-End Object Detection with Transformerscited by 7,396
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,021
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,498
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,417
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,820
- 2019Objects as Pointscited by 1,173
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 827
- 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 658
34 more
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 594
- 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 571
- 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 492
- 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 445
- 2021Conditional DETR for Fast Training Convergencecited by 423
- 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
- 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
- 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
- 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
- 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 203
- 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
- 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
- 2020RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Drivingcited by 150
- 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
- 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
- 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
- 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
- 2017Human Pose Regression by Combining Indirect Part Detection and Contextual Informationcited by 52
- 2024GeoDiffuser: Geometry-Based Image Editing with Diffusion Modelscited by 34
- 2025Detect Anything 3D in the Wildcited by 33
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
- 2024Open Vocabulary Monocular 3D Object Detectioncited by 23
- 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
- 2021MonoGRNet: A General Framework for Monocular 3D Object Detectioncited by 20
- 2024InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attentioncited by 13
- 2023SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variationscited by 12
- 2024PerLDiff: Controllable Street View Synthesis Using Perspective-Layout Diffusion Modelscited by 12
- 20253D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detectioncited by 11
- 2026InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generationcited by 8
- 2020CenterNet3D: An Anchor Free Object Detector for Point Cloudcited by 7
- 2025BoxDreamer: Dreaming Box Corners for Generalizable Object Pose Estimationcited by 3
- 2024CatFree3D: Category-agnostic 3D Object Detection with Diffusioncited by 2
- 2025MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulatorcited by 1
Built on by
Not cited yet.