mlsanityData to 14 Oct 2026

Guiding Diffusion with Deep Geometric Moments: Balancing Fidelity and Variation

arXiv 2505.12486 ↗First posted 18 May 2025cs.CVcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Text-to-image generation models have achieved remarkable capabilities in synthesizing images, but often struggle to provide fine-grained control over the output. Existing guidance approaches, such as segmentation maps and depth maps, introduce spatial rigidity that restricts the inherent diversity of diffusion models. In this work, we introduce Deep Geometric Moments (DGM) as a novel form of guidance that encapsulates the subject's visual features and nuances through a learned geometric prior. DGMs focus specifically on the subject itself compared to DINO or CLIP features, which suffer from overemphasis on global image features or semantics. Unlike ResNets, which are sensitive to pixel-wise perturbations, DGMs rely on robust geometric moments. Our experiments demonstrate that DGM effectively balance control and diversity in diffusion-based image generation, allowing a flexible control mechanism for steering the diffusion process.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 35 of its 37 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

35 of 37 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
23 more
  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  2. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  3. 2023Universal Guidance for Diffusion Modelscited by 437
  4. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  5. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 294
  6. 2024A Survey on Diffusion Models for Inverse Problemscited by 180
  7. 2024Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealingcited by 163
  8. 2023Manifold Preserving Guided Diffusioncited by 157
  9. 2024TFG: Unified Training-Free Guidance for Diffusion Modelscited by 107
  10. 2024No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Modelscited by 53
  11. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  12. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51
  13. 2024Steering Rectified Flow Models in the Vector Field for Controlled Image Generationcited by 39
  14. 2024Understanding and Improving Training-free Loss-based Diffusion Guidancecited by 32
  15. 2023ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generationscited by 24
  16. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 24
  17. 2025Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posteriorcited by 22
  18. 2025Distilling Diversity and Control in Diffusion Modelscited by 22
  19. 2025Training-Free Constrained Generation With Stable Diffusion Modelscited by 20
  20. 2023Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesiscited by 18
  21. 2024ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidancecited by 18
  22. 2024Flow Matching Posterior Sampling: A Training-free Conditional Generation for Flow Matchingcited by 4
  23. 2022Improving Shape Awareness and Interpretability in Deep Networks Using Geometric Momentscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
  2. 2026Repurposing Geometric Foundation Models for Multi-view Diffusion
  3. 2024Diffusion Models Need Visual Priors for Image Generation
  4. 2023Enhancing Diffusion Models with 3D Perspective Geometry Constraints
  5. 2026Video Generative Models as Geometry Learner
  6. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D
  7. 2023GECCO: Geometrically-Conditioned Point Diffusion Models
  8. 2024Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency