mlsanityData to 14 Oct 2026

Diffusing Differentiable Representations

arXiv 2412.06981 ↗First posted 9 Dec 2024cs.CV cs.LGcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce a novel, training-free method for sampling differentiable representations (diffreps) using pretrained diffusion models. Rather than merely mode-seeking, our method achieves sampling by "pulling back" the dynamics of the reverse-time process--from the image space to the diffrep parameter space--and updating the parameters according to this pulled-back process. We identify an implicit constraint on the samples induced by the diffrep and demonstrate that addressing this constraint significantly improves the consistency and detail of the generated objects. Our method yields diffreps with substantially improved quality and diversity for images, panoramas, and 3D NeRFs compared to existing techniques. Our approach is a general-purpose method for sampling diffreps, expanding the scope of problems that diffusion models can tackle.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 28 of its 33 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

28 of 33 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems13 Jun
  2. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  3. 2023Universal Guidance for Diffusion Models14 Feb
  4. 2022Classifier-Free Diffusion Guidance26 Jul
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 19

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  8. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  9. 2018Demystifying MMD GANscited by 1,293
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  12. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
7 more
  1. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 508
  2. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  3. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  4. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 110
  5. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  6. 2024Taming Stable Diffusion for Text to 360deg Panorama Image Generationcited by 12
  7. 2023CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebracited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PCM-NeRF: Probabilistic Camera Modeling for Neural Radiance Fields under Pose Uncertaintycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
  2. 2026Enhanced Low-Density Region Exploration in Classifier-Guided Diffusion Models Through Modified Reverse Diffusion Sampling
  3. 2024Diffusion Models in Low-Level Vision: A Survey
  4. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning
  5. 2024Diffusion Models Need Visual Priors for Image Generation
  6. 2026Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations
  7. 2026DiffRGD: An Inference-Time Diffusion Guidance Through Riemannian Gradient Descent
  8. 2025Diffusion differentiable resampling