# Score Distillation Sampling with Learned Manifold Corrective

arXiv: https://arxiv.org/abs/2401.05293 · HTML: https://mlsanity.com/paper/2401.05293/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 39 of its 41 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 39)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://mlsanity.com/paper/2305.08891/) (2023)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)

## Built on by (most cited first, 27 of 27)

- [Rethinking Score Distillation as a Bridge Between Image Distributions](https://mlsanity.com/paper/2406.09417/) (2024)
- [SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions](https://mlsanity.com/paper/2403.16627/) (2024)
- [InterGSEdit: Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Prior](https://mlsanity.com/paper/2507.04961/) (2025)
- [Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representation](https://mlsanity.com/paper/2409.03718/) (2024)
- [UniPart: Part-Level 3D Generation with Unified 3D Geom-Seg Latents](https://mlsanity.com/paper/2512.09435/) (2025)
- [PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World](https://mlsanity.com/paper/2605.05163/) (2026)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [GaussianFluent: Gaussian Simulation for Dynamic Scenes with Mixed Materials](https://mlsanity.com/paper/2601.09265/) (2026)
- [FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process](https://mlsanity.com/paper/2409.07451/) (2024)
- [DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mapping](https://mlsanity.com/paper/2409.05099/) (2024)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [Walking the Schr"odinger Bridge: A Direct Trajectory for Text-to-3D Generation](https://mlsanity.com/paper/2511.05609/) (2025)
- [Mean-Shift Distillation for Diffusion Mode Seeking](https://mlsanity.com/paper/2502.15989/) (2025)
- [Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculpting](https://mlsanity.com/paper/2502.20045/) (2025)
- [LUSD: Localized Update Score Distillation for Text-Guided Image Editing](https://mlsanity.com/paper/2503.11054/) (2025)
- [DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generation](https://mlsanity.com/paper/2507.02299/) (2025)
- [SV-GS: Sparse View 4D Reconstruction with Skeleton-Driven Gaussian Splatting](https://mlsanity.com/paper/2601.00285/) (2026)
- [Generative Video Compression with Adaptive Score Distillation](https://mlsanity.com/paper/2607.22772/) (2026)
- [UVRM: A Scalable 3D Reconstruction Model from Unposed Videos](https://mlsanity.com/paper/2501.09347/) (2025)
- [VEIGAR: View-consistent Explicit Inpainting and Geometry Alignment for 3D object Removal](https://mlsanity.com/paper/2506.15821/) (2025)
- [ConFixGS: Learning to Fix Feedforward 3D Gaussian Splatting with Confidence-Aware Diffusion Priors in Driving Scenes](https://mlsanity.com/paper/2605.09688/) (2026)
- [CAdam: Context-Adaptive Moment Estimation for 3D Gaussian Densification in Generative Distillation](https://mlsanity.com/paper/2605.20872/) (2026)
- [ISAP-3D: Identity-Slot Aligned Part-Aware 3D Generation](https://mlsanity.com/paper/2606.12099/) (2026)
- [SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation](https://mlsanity.com/paper/2609.03806/) (2026)
- [Learning 3D Editing without Paired Supervision via Generative Prior Distillation](https://mlsanity.com/paper/2609.04942/) (2026)

## Abstract (arXiv)

Score Distillation Sampling (SDS) is a recent but already widely popular method that relies on an image diffusion model to control optimization problems using text prompts. In this paper, we conduct an in-depth analysis of the SDS loss function, identify an inherent problem with its formulation, and propose a surprisingly easy but effective fix. Specifically, we decompose the loss into different factors and isolate the component responsible for noisy gradients. In the original formulation, high text guidance is used to account for the noise, leading to unwanted side effects such as oversaturation or repeated detail. Instead, we train a shallow network mimicking the timestep-dependent frequency bias of the image diffusion model in order to effectively factor it out. We demonstrate the versatility and the effectiveness of our novel loss formulation through qualitative and quantitative experiments, including optimization-based image synthesis and editing, zero-shot image translation network training, and text-to-3D synthesis.