Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping
Not on the map. Why?
Abstract (arXiv)
Autonomous agile robots need more than metric geometry: they must understand objects, rooms, places, and spatial relations for search, inspection, exploration, and human robot interaction. Conventional metric maps support localization and collision avoidance, but do not provide this semantic and relational structure. 3D scene graphs address this gap by connecting geometry with object level and room level understanding. Building such representations on agile platforms remains difficult because aerial and lightweight robots operate under strict payload, power, and compute limits, making RGB-D cameras and LiDAR sensors impractical for many onboard settings. We present Mono-Hydra++, a real time monocular RGB plus IMU pipeline for indoor metric semantic mapping and hierarchical 3D scene graph construction. The system combines M2H-MX, a DINOv3 based multi-task model for depth and semantics, with a deep feature visual inertial odometry front end, sparse predicted depth constraints in the VIO derived pose graph, semantic masking for dynamic regions, and pose aware temporal alignment before volumetric fusion in the Mono-Hydra backend. On the Go-SLAM ScanNet evaluation subset, Mono-Hydra++ achieves 1.6% lower average trajectory error than the strongest RGB-D baseline in our comparison, while using only monocular RGB plus IMU input. On calibrated 7-Scenes, it improves average ATE by 29.8% over the strongest competing calibrated baseline. We further validate Mono-Hydra++ in a real ITC building deployment using RealSense RGB plus IMU and demonstrate embedded feasibility by deploying the ONNX/TensorRT FP16 M2H-MX-L perception model at 25.53 FPS on a Jetson Orin NX 16GB. These results show that Mono-Hydra++ can provide real time metric semantic mapping and scene graph construction for resource constrained robotic platforms without relying on active depth sensors.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
47 of 57 references.
References · 47
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2021Vision Transformers for Dense Predictioncited by 1,803
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
- 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
- 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
35 more
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
- 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
- 2017VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimatorcited by 423
- 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 281
- 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
- 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
- 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
- 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 184
- 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
- 2023NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAMcited by 164
- 2022ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fieldscited by 158
- 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
- 2022Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representationcited by 149
- 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
- 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
- 2023GO-SLAM: Global Optimization for Consistent 3D Instant Reconstructioncited by 133
- 2022InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understandingcited by 89
- 2021Exploring Relational Context for Multi-Task Dense Predictioncited by 71
- 2024Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussianscited by 66
- 2024Multi-Task Dense Prediction via Mixture of Low-Rank Expertscited by 58
- 2018CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRFcited by 48
- 2021AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentationcited by 31
- 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
- 2019DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraintscited by 27
- 2024MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoderscited by 22
- 2024SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Imagescited by 13
- 2024MTMamba++: Enhancing Multi-Task Dense Scene Understanding via Mamba-Based Decoderscited by 10
- 2018Robocentric Visual-Inertial Odometrycited by 7
- 2023Mono-hydra: Real-time 3D scene graph construction from monocular camera input with IMUcited by 4
- 2024Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learningcited by 4
- 2025SP-SLAM: Neural Real-Time Dense SLAM With Scene Priorscited by 4
- 2026VGGT-SLAM++cited by 4
- 2025M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial Perceptioncited by 3
- 2022Efficient Video Segmentation Models with Per-frame Inferencecited by 1
- 2026M2H-MX: Multi-Task Semantic and Geometric Perception for Real-Time Monocular 3D Scene Graph Constructioncited by 1
Built on by
Not cited yet.