mlsanityData to 14 Oct 2026

Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis

arXiv 2208.13753 ↗First posted 29 Aug 2022cs.CV cs.AI cs.LGcited by 75

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models (DMs) have shown great potential for high-quality image synthesis. However, when it comes to producing images with complex scenes, how to properly describe both image global structures and object details remains a challenging task. In this paper, we present Frido, a Feature Pyramid Diffusion model performing a multi-scale coarse-to-fine denoising process for image synthesis. Our model decomposes an input image into scale-dependent vector quantized features, followed by a coarse-to-fine gating for producing image output. During the above multi-scale representation learning stage, additional input conditions like text, scene graph, or image layout can be further exploited. Thus, Frido can be also applied for conditional or cross-modality image synthesis. We conduct extensive experiments over various unconditioned and conditional image generation tasks, ranging from text-to-image synthesis, layout-to-image, scene-graph-to-image, to label-to-image. More specifically, we achieved state-of-the-art FID scores on five benchmarks, namely layout-to-image on COCO and OpenImages, scene-graph-to-image on COCO and Visual Genome, and label-to-image on COCO. Code is available at https://github.com/davidhalladay/Frido.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 78 of its 99 references that are in the map. First posted on arXiv 29 Aug 2022.

Builds on

78 of 99 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022Improved Vector Quantized Diffusion Models31 May
  2. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents2 Jan
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  5. 2021Score-based Generative Modeling in Latent Space10 Jun
  6. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation10 Jun
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 68

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2017Neural Discrete Representation Learningcited by 4,712
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
38 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  8. 2016Density estimation using Real NVPcited by 2,042
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  10. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,630
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  12. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  13. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  14. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  15. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  16. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  17. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  18. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  19. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  20. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  21. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  22. 2016Pixel Recurrent Neural Networkscited by 816
  23. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  24. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  25. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  26. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  27. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  28. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  29. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  30. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  31. 2018Assessing Generative Models via Precision and Recallcited by 406
  32. 2018Image Generation from Scene Graphscited by 324
  33. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  34. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  35. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  36. 2021Projected GANs Converge Fastercited by 208
  37. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 191
  38. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 165

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 29; 2024: 22; 2025: 14; 2026: 4.

75 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  3. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  4. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  5. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  6. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
  7. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  8. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  9. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 179
  10. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Modelscited by 155
  11. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  12. 2023Freestyle Layout-to-Image Synthesiscited by 67
38 more
  1. 2023Pyramid Diffusion Models For Low-light Image Enhancementcited by 67
  2. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Promptscited by 54
  3. 2022Efficient Diffusion Models for Vision: A Surveycited by 51
  4. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesiscited by 51
  5. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 50
  6. 2022X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusioncited by 49
  7. 2023DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimationcited by 36
  8. 2023One-shot Unsupervised Domain Adaptation with Personalized Diffusion Modelscited by 35
  9. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Datacited by 33
  10. 2024Improving Diffusion-Based Image Synthesis with Context Predictioncited by 32
  11. 2024DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perceptioncited by 32
  12. 2023SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Modelscited by 30
  13. 2023Speech Driven Video Editing via an Audio-Conditioned Diffusion Modelcited by 28
  14. 2024Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentationcited by 27
  15. 2023ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connectioncited by 25
  16. 2023LLMGA: Multimodal Large Language Model based Generation Assistantcited by 25
  17. 2024Revisiting Non-Autoregressive Transformers for Efficient Image Synthesiscited by 22
  18. 2025The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencodingcited by 22
  19. 2023Context Diffusion: In-Context Aware Image Generationcited by 21
  20. 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layoutscited by 20
  21. 2023DiffTAD: Temporal Action Detection with Proposal Denoising Diffusioncited by 19
  22. 2024Desigen: A Pipeline for Controllable Design Template Generationcited by 19
  23. 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Modelscited by 18
  24. 2023iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Viewscited by 14
  25. 2023Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generationcited by 13
  26. 2024Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoisercited by 13
  27. 2024AdaNAT: Exploring Adaptive Policy for Token-Based Image Generationcited by 13
  28. 2024Discriminative Probing and Tuning for Text-to-Image Generationcited by 11
  29. 2023Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generatorcited by 9
  30. 2023Transformer-based Image Generation from Scene Graphscited by 8
  31. 2023Vision + Language Applications: A Surveycited by 8
  32. 2024ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesiscited by 8
  33. 2024Bring Metric Functions into Diffusion Modelscited by 7
  34. 2023Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Modelscited by 6
  35. 2025DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modelingcited by 6
  36. 2024BlindDiff: Empowering Degradation Modelling in Diffusion Models for Blind Image Super-Resolutioncited by 5
  37. 2024CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimatorscited by 5
  38. 2025Scalable GANs with Transformerscited by 5

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Conditional Image Synthesis with Diffusion Models: A Survey
  2. 2023Patched Denoising Diffusion Models For High-Resolution Image Synthesis
  3. 2024Diffusion Model-Based Image Editing: A Survey
  4. 2023Pyramid Diffusion for Fine 3D Large Scene Generation
  5. 2024Ultra-High-Resolution Image Synthesis with Pyramid Diffusion Model
  6. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
  7. 2022Pyramidal Denoising Diffusion Probabilistic Models
  8. 2024Diffusion Models in Low-Level Vision: A Survey