mlsanityData to 7 Oct 2026

Score Distillation Sampling with Learned Manifold Corrective

arXiv 2401.05293 ↗First posted 10 Jan 2024cs.CVcited by 27

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Score Distillation Sampling (SDS) is a recent but already widely popular method that relies on an image diffusion model to control optimization problems using text prompts. In this paper, we conduct an in-depth analysis of the SDS loss function, identify an inherent problem with its formulation, and propose a surprisingly easy but effective fix. Specifically, we decompose the loss into different factors and isolate the component responsible for noisy gradients. In the original formulation, high text guidance is used to account for the noise, leading to unwanted side effects such as oversaturation or repeated detail. Instead, we train a shallow network mimicking the timestep-dependent frequency bias of the image diffusion model in order to effectively factor it out. We demonstrate the versatility and the effectiveness of our novel loss formulation through qualitative and quantitative experiments, including optimization-based image synthesis and editing, zero-shot image translation network training, and text-to-3D synthesis.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 39 of its 41 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

39 of 41 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  2. 2023Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views27 Aug
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  7. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  8. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
19 more
  1. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  2. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  3. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  4. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  5. 2022Diffusion Models in Vision: A Surveycited by 1,008
  6. 2022Denoising Diffusion Restoration Modelscited by 951
  7. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  8. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  9. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  10. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  11. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  12. 2023Implicit Diffusion Models for Continuous Super-Resolutioncited by 216
  13. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  14. 2023Delta Denoising Scorecited by 120
  15. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  16. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  17. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  18. 2023Noise-Free Score Distillationcited by 99
  19. 2023Collaborative Score Distillation for Consistent Visual Synthesiscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 12; 2026: 9.

27 papers.

Sort:

By area:

  1. 2024Rethinking Score Distillation as a Bridge Between Image Distributionscited by 36
  2. 2024SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditionscited by 23
  3. 2025InterGSEdit: Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Priorcited by 19
  4. 2024Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with Image-Based Surface Representationcited by 17
  5. 2025UniPart: Part-Level 3D Generation with Unified 3D Geom-Seg Latentscited by 9
  6. 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 8
  7. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  8. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  9. 2024FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Processcited by 6
  10. 2024DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mappingcited by 5
  11. 2024Data Augmentation via Latent Diffusion for Saliency Predictioncited by 5
  12. 2026GaussianFluent: Gaussian Simulation for Dynamic Scenes with Mixed Materialscited by 5
15 more
  1. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  2. 2025Walking the Schr"odinger Bridge: A Direct Trajectory for Text-to-3D Generationcited by 3
  3. 2025Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculptingcited by 1
  4. 2025LUSD: Localized Update Score Distillation for Text-Guided Image Editingcited by 1
  5. 2025DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generationcited by 1
  6. 2026SV-GS: Sparse View 4D Reconstruction with Skeleton-Driven Gaussian Splattingcited by 1
  7. 2026Generative Video Compression with Adaptive Score Distillationcited by 1
  8. 2025UVRM: A Scalable 3D Reconstruction Model from Unposed Videoscited by 0
  9. 2025Mean-Shift Distillation for Diffusion Mode Seekingcited by 0
  10. 2025VEIGAR: View-consistent Explicit Inpainting and Geometry Alignment for 3D object Removalcited by 0
  11. 2026ConFixGS: Learning to Fix Feedforward 3D Gaussian Splatting with Confidence-Aware Diffusion Priors in Driving Scenescited by 0
  12. 2026CAdam: Context-Adaptive Moment Estimation for 3D Gaussian Densification in Generative Distillationcited by 0
  13. 2026ISAP-3D: Identity-Slot Aligned Part-Aware 3D Generationcited by 0
  14. 2026SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generationcited by 0
  15. 2026Learning 3D Editing without Paired Supervision via Generative Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D
  2. 2025RewardSDS: Aligning Score Distillation via Reward-Weighted Sampling
  3. 2025Few-Step Diffusion via Score identity Distillation
  4. 2024Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection
  5. 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation
  6. 2024Score Distillation via Reparametrized DDIM
  7. 2024Inference-Time Diffusion Model Distillation
  8. 2024DreamSampler: Unifying Diffusion Sampling and Score Distillation for Image Manipulation