# Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regression

arXiv: https://arxiv.org/abs/2205.04565 · HTML: https://mlsanity.com/paper/2205.04565/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 59 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Robust Consistent Video Depth Estimation](https://mlsanity.com/paper/2012.05901/) (2020)
- [Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging](https://mlsanity.com/paper/2105.14021/) (2021)
- [Guiding Monocular Depth Estimation Using Depth-Attention Volume](https://mlsanity.com/paper/2004.02760/) (2020)
- [Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.03650/) (2016)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)
- [SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation](https://mlsanity.com/paper/1810.01849/) (2018)
- [DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning](https://mlsanity.com/paper/2003.13446/) (2020)
- [LEGO: Learning Edge with Geometry all at Once by Watching Videos](https://mlsanity.com/paper/1803.05648/) (2018)
- [What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?](https://mlsanity.com/paper/1801.06397/) (2018)
- [StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimation](https://mlsanity.com/paper/2109.10115/) (2021)
- [R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes](https://mlsanity.com/paper/2108.04814/) (2021)
- [Depth from a polarisation + RGB stereo pair](https://mlsanity.com/paper/1903.12061/) (2019)
- [Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data](https://mlsanity.com/paper/2008.01034/) (2020)
- [Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2110.08192/) (2021)
- [SteReFo: Efficient Image Refocusing with Stereo Vision](https://mlsanity.com/paper/1909.13395/) (2019)

## Built on by (most cited first, 19 of 19)

- [UniDepth: Universal Monocular Metric Depth Estimation](https://mlsanity.com/paper/2403.18913/) (2024)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [HouseCat6D -- A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios](https://mlsanity.com/paper/2212.10428/) (2022)
- [Masked Depth Modeling for Spatial Perception](https://mlsanity.com/paper/2601.17895/) (2026)
- [MonoGraspNet: 6-DoF Grasping with a Single RGB Image](https://mlsanity.com/paper/2209.13036/) (2022)
- [TexPose: Neural Texture Learning for Self-Supervised 6D Object Pose Estimation](https://mlsanity.com/paper/2212.12902/) (2022)
- [Integrating Extra Modality Helps Segmentor Find Camouflaged Objects Well](https://mlsanity.com/paper/2502.14471/) (2025)
- [PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation](https://mlsanity.com/paper/2607.02515/) (2026)
- [Robust Depth Enhancement via Polarization Prompt Fusion Tuning](https://mlsanity.com/paper/2404.04318/) (2024)
- [PolarAnything: Diffusion-based Polarimetric Image Synthesis](https://mlsanity.com/paper/2507.17268/) (2025)
- [Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment](https://mlsanity.com/paper/2208.03167/) (2022)
- [S2P3: Self-Supervised Polarimetric Pose Prediction](https://mlsanity.com/paper/2312.01105/) (2023)
- [The Midas Touch for Metric Depth](https://mlsanity.com/paper/2605.11578/) (2026)
- [Boosting Monocular Metric Depth Estimation via Bokeh Rendering](https://mlsanity.com/paper/2512.12425/) (2025)
- [Multi-Modal Dataset Acquisition for Photometrically Challenging Object](https://mlsanity.com/paper/2308.10621/) (2023)
- [Zero-Shot Depth from Defocus](https://mlsanity.com/paper/2603.26658/) (2026)
- [CDPR: Cross-modal Diffusion with Polarization for Reliable Monocular Depth Estimation](https://mlsanity.com/paper/2604.11097/) (2026)
- [Streaming Neural Speech Codecs through Time-Invariant Representations](https://mlsanity.com/paper/2607.05250/) (2026)
- [The Midas Touch for Metric Depth: Interpretable Metricization for Generalizable Depth Perception](https://mlsanity.com/paper/2610.16749/) (2026)

## Abstract (arXiv)

Depth estimation is a core task in 3D computer vision. Recent methods investigate the task of monocular depth trained with various depth sensor modalities. Every sensor has its advantages and drawbacks caused by the nature of estimates. In the literature, mostly mean average error of the depth is investigated and sensor capabilities are typically not discussed. Especially indoor environments, however, pose challenges for some devices. Textureless regions pose challenges for structure from motion, reflective materials are problematic for active sensing, and distances for translucent material are intricate to measure with existing sensors. This paper proposes HAMMER, a dataset comprising depth estimates from multiple commonly used sensors for indoor depth estimation, namely ToF, stereo, structured light together with monocular RGB+P data. We construct highly reliable ground truth depth maps with the help of 3D scanners and aligned renderings. A popular depth estimators is trained on this data and typical depth senosors. The estimates are extensively analyze on different scene structures. We notice generalization issues arising from various sensor technologies in household environments with challenging but everyday scene content. HAMMER, which we make publicly available, provides a reliable base to pave the way to targeted depth improvements and sensor fusion approaches.