# 3DRot: Rediscovering the Missing Primitive for RGB-Based 3D Augmentation

arXiv: https://arxiv.org/abs/2508.01423 · HTML: https://mlsanity.com/paper/2508.01423/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 2 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild](https://mlsanity.com/paper/2207.10660/) (2022)
- [MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships](https://mlsanity.com/paper/2003.00504/) (2020)
- [DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding](https://mlsanity.com/paper/2411.14347/) (2024)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [UniK3D: Universal Camera Monocular 3D Estimation](https://mlsanity.com/paper/2503.16591/) (2025)
- [Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detection](https://mlsanity.com/paper/2205.09373/) (2022)
- [The Devil is in the Task: Exploiting Reciprocal Appearance-Localization Features for Monocular 3D Object Detection](https://mlsanity.com/paper/2112.14023/) (2021)
- [OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objects](https://mlsanity.com/paper/2407.08711/) (2024)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [MonoLSS: Learnable Sample Selection For Monocular 3D Detection](https://mlsanity.com/paper/2312.14474/) (2023)
- [Exploring Geometric Consistency for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.05858/) (2021)
- [3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection](https://mlsanity.com/paper/2312.05277/) (2023)
- [Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D](https://mlsanity.com/paper/2403.18922/) (2024)
- [Two-level Data Augmentation for Calibrated Multi-view Detection](https://mlsanity.com/paper/2210.10756/) (2022)
- [CARLA Drone: Monocular 3D Object Detection from a Different Perspective](https://mlsanity.com/paper/2408.11958/) (2024)
- [How Do Images Align and Complement LiDAR? Towards a Harmonized Multi-modal 3D Panoptic Segmentation](https://mlsanity.com/paper/2505.18956/) (2025)
- [MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection](https://mlsanity.com/paper/2504.06801/) (2025)

## Abstract (arXiv)

RGB-based 3D tasks, e.g., 3D detection, depth estimation, 3D keypoint estimation, still suffer from scarce, expensive annotations and a thin augmentation toolbox, since many image transforms, including rotations and warps, disrupt geometric consistency. While horizontal flipping and color jitter are standard, rigorous 3D rotation augmentation has surprisingly remained absent from RGB-based pipelines, largely due to the misconception that it requires scene depth or scene reconstruction. In this paper, we introduce 3DRot, a plug-and-play augmentation that rotates and mirrors images about the camera's optical center while synchronously updating RGB images, camera intrinsics, object poses, and 3D annotations to preserve projective geometry, achieving geometry-consistent rotations and reflections without relying on any scene depth. We first validate 3DRot on a classical RGB-based 3D task, monocular 3D detection. On SUN RGB-D, inserting 3DRot into a frozen DINO-X + Cube R-CNN pipeline raises IoU3D from 43.21 to 44.51, cuts rotation error (ROT) from 22.91^circ to 20.93^circ, and boosts mAP0.5 from 35.70 to 38.11; smaller but consistent gains appear on a cross-domain IN10 split. Beyond monocular detection, adding 3DRot on top of the standard BTS augmentation schedule further improves NYU Depth v2 from 0.1783 to 0.1685 in abs-rel (and 0.7472 to 0.7548 in δ<1.25), and reduces cross-dataset error on SUN RGB-D. On KITTI, applying the same camera-centric rotations in MVX-Net (LiDAR+RGB) raises moderate 3D AP from about 63.85 to 65.16 while remaining compatible with standard 3D augmentations.