mlsanityData to 14 Oct 2026

Generating High Fidelity Data from Low-density Regions using Diffusion Models

arXiv 2203.17260 ↗First posted 31 Mar 2022cs.CV cs.LGcited by 78

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Our work focuses on addressing sample deficiency from low-density regions of data manifold in common image datasets. We leverage diffusion process based generative models to synthesize novel images from low-density regions. We observe that uniform sampling from diffusion models predominantly samples from high-density regions of the data manifold. Therefore, we modify the sampling process to guide it towards low-density regions while simultaneously maintaining the fidelity of synthetic data. We rigorously demonstrate that our process successfully generates novel high fidelity samples from low-density regions. We further examine generated samples and show that the model does not memorize low-density data and indeed learns to generate novel samples from low-density regions.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 41 of its 49 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

41 of 49 references.

Closest ancestors · 9

Same direction, newest first

  1. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  2. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  4. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun
  9. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 32

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
20 more
  1. 2019Natural Adversarial Examplescited by 1,408
  2. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  4. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  6. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
  7. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  8. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  9. 2018Assessing Generative Models via Precision and Recallcited by 406
  10. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  11. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  12. 2021Improving Robustness using Generated Datacited by 234
  13. 2019PyOD: A Python Toolbox for Scalable Outlier Detectioncited by 222
  14. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  15. 2020Estimating Example Difficulty Using Variance of Gradientscited by 97
  16. 2018Discriminator Rejection Samplingcited by 49
  17. 2021Densely connected normalizing flowscited by 49
  18. 2020Instance Selection for GANscited by 23
  19. 2019Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space with Softplus Losscited by 11
  20. 2020Time for a Background Check! Uncovering the impact of Background Features on Deep Neural Networkscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 25; 2024: 19; 2025: 17; 2026: 10.

78 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2022Diffusion Models in Vision: A Surveycited by 1,015
  3. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  4. 2022Diffusion Models already have a Semantic Latent Spacecited by 328
  5. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 129
  6. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 125
  7. 2022Exploring Chemical Space with Score-based Out-of-distribution Generationcited by 95
  8. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  9. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplerscited by 74
  10. 2023On the Generalization Properties of Diffusion Modelscited by 68
  11. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  12. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistentcited by 58
38 more
  1. 2024Towards Realistic Data Generation for Real-World Super-Resolutioncited by 56
  2. 2023Class-Balancing Diffusion Modelscited by 48
  3. 2023DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Modelscited by 43
  4. 2023Don't Play Favorites: Minority Guidance for Diffusion Modelscited by 37
  5. 2023Diffusion-based Conditional ECG Generation with Structured State Space Modelscited by 36
  6. 2023Training-free Content Injection using h-space in Diffusion Modelscited by 36
  7. 2024Consistency-diversity-realism Pareto fronts of conditional image generative modelscited by 36
  8. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  9. 2024MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Processcited by 32
  10. 2023ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editingcited by 30
  11. 2024Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budgetcited by 30
  12. 2023Unsupervised Discovery of Semantic Latent Directions in Diffusion Modelscited by 28
  13. 2022Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Modelscited by 27
  14. 2024Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidancecited by 24
  15. 2023Distribution Shift Inversion for Out-of-Distribution Predictioncited by 23
  16. 2025Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentationcited by 23
  17. 2024Image Generation Diversity Issues and How to Tame Themcited by 22
  18. 2024Self-Guided Generation of Minority Samples Using Diffusion Modelscited by 19
  19. 2024Minority-Focused Text-to-Image Generation via Prompt Optimizationcited by 18
  20. 2023AdaDiff: Accelerating Diffusion Models through Step-Wise Adaptive Computationcited by 14
  21. 2025SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Scorecited by 14
  22. 2024ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generationcited by 13
  23. 2023From Posterior Sampling to Meaningful Diversity in Image Restorationcited by 12
  24. 2025Diverse Text-to-Image Generation via Contrastive Noise Optimizationcited by 12
  25. 2022Part-Based Models Improve Adversarial Robustnesscited by 11
  26. 2024TextSSR: Diffusion-based Data Synthesis for Scene Text Recognitioncited by 8
  27. 2023PaintSeg: Training-free Segmentation via Paintingcited by 7
  28. 2023Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Modelscited by 6
  29. 2024Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMscited by 6
  30. 2024Joint Diffusion models in Continual Learningcited by 6
  31. 2025Increasing the Utility of Synthetic Images through Chamfer Guidancecited by 6
  32. 2024GUIDE: Guidance-based Incremental Learning with Diffusion Modelscited by 5
  33. 2025Devil is in the Details: Density Guidance for Detail-Aware Generation with Flow Modelscited by 5
  34. 2025Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generationcited by 5
  35. 2024Hierarchical Uncertainty Exploration via Feedforward Posterior Treescited by 4
  36. 2025DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Modelscited by 4
  37. 2025Diverse Video Generation with Determinantal Point Process-Guided Policy Optimizationcited by 4
  38. 2023Uncertainty Visualization via Low-Dimensional Posterior Projectionscited by 3

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative Learning of Densities on Manifolds
  2. 2026Diffusion Image Generation with Explicit Modeling of Data Manifold Geometry
  3. 2024Non-Normal Diffusion Models
  4. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions
  5. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  6. 2022Representation Learning with Diffusion Models
  7. 2025MAD: Manifold Attracted Diffusion
  8. 2024Diffusion Models in Low-Level Vision: A Survey