mlsanityData to 7 Oct 2026

T1: Scaling Diffusion Probabilistic Fields to High-Resolution on Unified Visual Modalities

arXiv 2305.14674 ↗First posted 24 May 2023cs.CVcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion Probabilistic Field (DPF) models the distribution of continuous functions defined over metric spaces. While DPF shows great potential for unifying data generation of various modalities including images, videos, and 3D geometry, it does not scale to a higher data resolution. This can be attributed to the ``scaling property'', where it is difficult for the model to capture local structures through uniform sampling. To this end, we propose a new model comprising of a view-wise sampling algorithm to focus on local structure learning, and incorporating additional guidance, e.g., text description, to complement the global geometry. The model can be scaled to generate high-resolution data while unifying multiple modalities. Experimental results on data generation in various modalities demonstrate the effectiveness of our model, as well as its potential as a foundation framework for scalable modality-unified visual content generation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 53 of its 61 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

53 of 61 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Diffusion Probabilistic Fields1 Mar
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Probabilistic Models19 Jun
  7. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2013Auto-Encoding Variational Bayescited by 3,980
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
34 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  12. 2016HyperNetworkscited by 1,003
  13. 2016Least Squares Generative Adversarial Networkscited by 1,000
  14. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  15. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  16. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  17. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  18. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  19. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  20. 2018Assessing Generative Models via Precision and Recallcited by 401
  21. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  22. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  23. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  24. 2022From data to functa: Your data point is a function and you can treat it like onecited by 183
  25. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  26. 2019Convolutional Conditional Neural Processescited by 139
  27. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
  28. 2021Generative Models as Distributions of Functionscited by 89
  29. 2022ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformerscited by 70
  30. 2023Spatial Functa: Scaling Functa to ImageNet Classification and Generationcited by 55
  31. 2022Neural Diffusion Processescited by 50
  32. 2021Learning Signal-Agnostic Manifolds of Neural Fieldscited by 45
  33. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  34. 2021Ethics and Creativity in Computer Visioncited by 15

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Bigger is not Always Better: Scaling Properties of Latent Diffusion Modelscited by 21

Similar, not cited

Close in wording, no citation either way

  1. 2023Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis
  2. 2026Abra: Scaling Diffusion Image Training
  3. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  4. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data
  5. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models
  6. 2024Diffusion Models Need Visual Priors for Image Generation
  7. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning
  8. 2024Scaling Properties of Diffusion Models for Perceptual Tasks