Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
Recovering pixel-wise geometric properties from a single image is fundamentally ill-posed due to appearance ambiguity and non-injective mappings between 2D observations and 3D structures. While discriminative regression models achieve strong performance through large-scale supervision, their success is bounded by the scale, quality, and diversity of available data, as well as by limited physical reasoning. Recent diffusion models exhibit powerful world priors that encode geometry and semantics learned from massive image-text data, yet directly reusing their stochastic generative formulation is suboptimal for deterministic geometric inference: the former is optimized for diverse and high-fidelity image generation, whereas the latter requires stable and accurate predictions. In this work, we propose Lotus-2, a two-stage deterministic framework for stable, accurate and fine-grained geometric dense prediction, aiming to provide an optimal adaptation protocol to fully exploit the pre-trained generative priors. Specifically, in the first stage, the core predictor employs a single-step deterministic formulation with a clean-data objective and a lightweight local continuity module (LCM) to generate globally coherent structures without grid artifacts. In the second stage, the detail sharpener performs a constrained multi-step rectified-flow refinement within the manifold defined by the core predictor, enhancing fine-grained geometry through noise-free deterministic flow matching. Using only 59K training samples, less than 1% of existing large-scale datasets, Lotus-2 establishes new state-of-the-art results in monocular depth estimation and highly competitive surface normal prediction. These results demonstrate that diffusion models can serve as deterministic world priors, enabling high-quality geometric reasoning beyond traditional discriminative and generative paradigms.
Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 59 of its 79 references that are in the map. First posted on arXiv 30 Nov 2025.
Builds on
59 of 79 references.
Closest ancestors · 6
Same direction, newest first
- 2025Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimation20 Mar
- 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think17 Sep
- 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal24 Jun
- 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar
- 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
Other references · 53
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2017Neural Discrete Representation Learningcited by 4,650
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
38 more
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2016Gaussian Error Linear Units (GELUs)cited by 3,431
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2024Depth Anything V2cited by 1,385
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
- 2021Alias-Free Generative Adversarial Networkscited by 1,181
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
- 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
- 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
- 2020Virtual KITTI 2cited by 345
- 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
- 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
- 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
- 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
- 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151
- 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 137
- 2024Rethinking Inductive Biases for Surface Normal Estimationcited by 122
- 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 58
- 2021DIML/CVL RGB-D Dataset: 2M RGB-D Images of Natural Indoor and Outdoor Scenescited by 45
- 2023Exploiting Diffusion Prior for Generalizable Dense Predictioncited by 37
- 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
- 2025DA2: Depth Anything in Any Directioncited by 22
- 2024DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generationcited by 13
- 2024Bi-TTA: Bidirectional Test-Time Adapter for Remote Physiological Measurementcited by 6
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 16.
16 papers.
Sort:
By area:
- 2026Image Generators are Generalist Vision Learnerscited by 40
- 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 28
- 2026Video Generation Models are General-Purpose Vision Learnerscited by 12
- 2026Vision as Unified Multimodal Generationcited by 5
- 2026DVD: Deterministic Video Depth Estimation with Generative Priorscited by 3
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstructioncited by 3
- 2026AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real Worldcited by 2
- 2026DepthART: Scaling Foundation Monocular Depth to Tiny Modelscited by 1
- 2026PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimationcited by 1
- 2026Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attentioncited by 1
- 2026Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluationcited by 1
- 2026Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimationcited by 0
4 more
- 2026MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Predictioncited by 0
- 2026Let RGB Be the Language of Visioncited by 0
- 2026TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimationcited by 0
- 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimationcited by 0
- 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation8 Sep 2026
- 2026Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation7 Sep 2026
- 2026TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimation6 Sep 2026
- 2026Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention31 Aug 2026
- 2026PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation17 Aug 2026
- 2026DepthART: Scaling Foundation Monocular Depth to Tiny Models19 Jul 2026
- 2026Let RGB Be the Language of Vision14 Jul 2026
- 2026Video Generation Models are General-Purpose Vision Learners10 Jul 2026
- 2026Vision as Unified Multimodal Generation7 Jul 2026
- 2026MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Prediction29 Jun 2026
- 2026AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World29 Jun 2026
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction19 Jun 2026
4 more
- 2026Image Generators are Generalist Vision Learners22 Apr 2026
- 2026Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation29 Mar 2026
- 2026DVD: Deterministic Video Depth Estimation with Generative Priors12 Mar 2026
- 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion8 Feb 2026
- 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 28
- 2026MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Predictioncited by 0
- 2026TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimationcited by 0
- 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimationcited by 0
- 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation8 Sep 2026
- 2026TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimation6 Sep 2026
- 2026MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Prediction29 Jun 2026
- 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion8 Feb 2026
- 2026Image Generators are Generalist Vision Learnerscited by 40
- 2026Video Generation Models are General-Purpose Vision Learnerscited by 12
- 2026Vision as Unified Multimodal Generationcited by 5
- 2026DVD: Deterministic Video Depth Estimation with Generative Priorscited by 3
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstructioncited by 3
- 2026AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real Worldcited by 2
- 2026DepthART: Scaling Foundation Monocular Depth to Tiny Modelscited by 1
- 2026PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimationcited by 1
- 2026Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attentioncited by 1
- 2026Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluationcited by 1
- 2026Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimationcited by 0
- 2026Let RGB Be the Language of Visioncited by 0
- 2026Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation7 Sep 2026
- 2026Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention31 Aug 2026
- 2026PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation17 Aug 2026
- 2026DepthART: Scaling Foundation Monocular Depth to Tiny Models19 Jul 2026
- 2026Let RGB Be the Language of Vision14 Jul 2026
- 2026Video Generation Models are General-Purpose Vision Learners10 Jul 2026
- 2026Vision as Unified Multimodal Generation7 Jul 2026
- 2026AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World29 Jun 2026
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction19 Jun 2026
- 2026Image Generators are Generalist Vision Learners22 Apr 2026
- 2026Poppy: Polarization-based Plug-and-Play Guidance for Enhancing Monocular Normal Estimation29 Mar 2026
- 2026DVD: Deterministic Video Depth Estimation with Generative Priors12 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025From Editor to Dense Geometry Estimator
- 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
- 2026Video Generative Models as Geometry Learner
- 2025D3-Predictor: Noise-Free Deterministic Diffusion for Dense Prediction
- 2026Repurposing Geometric Foundation Models for Multi-view Diffusion
- 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridging
- 2026PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion
- 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior