mlsanityData to 7 Oct 2026

Collaborative Score Distillation for Consistent Visual Synthesis

arXiv 2307.04787 ↗First posted 4 Jul 2023cs.CV cs.LGcited by 23

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e.g., video), achieving consistency across a set of images is challenging. In this paper, we address this challenge with a novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as "particles" in the SVGD update and combine their score functions to distill generative priors over a set of images synchronously. Thus, CSD facilitates seamless integration of information across 2D images, leading to a consistent visual synthesis across multiple samples. We show the effectiveness of CSD in a variety of tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 55 of its 62 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

55 of 62 references.

Closest ancestors · 14

Same direction, newest first

  1. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  3. 2022Diffusion Guided Domain Adaptation of Image Generators8 Dec
  4. 2022Sketch-Guided Text-to-Image Diffusion Models24 Nov
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  8. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  11. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
2 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
29 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  7. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  8. 2021Variational Diffusion Modelscited by 1,145
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  12. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  13. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  14. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  15. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  16. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  17. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  18. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  19. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  20. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  21. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  22. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  23. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  24. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  25. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 172
  26. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  27. 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
  28. 2023Delta Denoising Scorecited by 120
  29. 2017Measuring Sample Quality with Kernelscited by 119

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 15; 2025: 0; 2026: 4.

23 papers.

Sort:

By area:

  1. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  2. 2023A Survey on Video Diffusion Modelscited by 203
  3. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  4. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  5. 2024Learning the 3D Fauna of the Webcited by 58
  6. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  7. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
  8. 2024InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differencescited by 34
  9. 2024DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editingcited by 28
  10. 2024Score Distillation Sampling with Learned Manifold Correctivecited by 27
  11. 2024DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flowcited by 26
  12. 2023As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priorscited by 24
11 more
  1. 2024Hierarchical Patch Diffusion Models for High-Resolution Video Generationcited by 24
  2. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
  3. 2023Adversarial Score Distillation: When score distillation meets GANcited by 15
  4. 2024DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservationcited by 13
  5. 2026VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformerscited by 8
  6. 2024DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Modelscited by 5
  7. 2024Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversioncited by 4
  8. 2024Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injectioncited by 4
  9. 2026Stroke of Surprise: Progressive Semantic Illusions in Vector Sketchingcited by 1
  10. 2026EditCrafter: Tuning-free High-Resolution Image Editing via Pretrained Diffusion Modelcited by 0
  11. 2026JanusMesh: Fast and Zero-Shot 3D Visual Illusion Generation via Cross-Space Denoisingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking Score Distillation as a Bridge Between Image Distributions
  2. 2025LUSD: Localized Update Score Distillation for Text-Guided Image Editing
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
  4. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency
  5. 2026Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D
  6. 2024Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps
  7. 2026VISTA: Test-Time Compositional Alignment for Visual Autoregressive Generation
  8. 2025Few-Step Diffusion via Score identity Distillation