mlsanityData to 14 Oct 2026

Condition-Aware Neural Network for Controlled Image Generation

arXiv 2404.01143 ↗First posted 1 Apr 2024cs.CV cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

We present Condition-Aware Neural Network (CAN), a new method for adding control to image generative models. In parallel to prior conditional control methods, CAN controls the image generation process by dynamically manipulating the weight of the neural network. This is achieved by introducing a condition-aware weight generation module that generates conditional weight for convolution/linear layers based on the input condition. We test CAN on class-conditional image generation on ImageNet and text-to-image generation on COCO. CAN consistently delivers significant improvements for diffusion transformer models, including DiT and UViT. In particular, CAN combined with EfficientViT (CaT) achieves 2.78 FID on ImageNet 512x512, surpassing DiT-XL/2 while requiring 52x fewer MACs per sampling step.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 32 of its 37 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

32 of 37 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  11. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
20 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  7. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  8. 2023Consistency Modelscited by 1,822
  9. 2016HyperNetworkscited by 1,027
  10. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  12. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  13. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
  14. 2018Slimmable Neural Networkscited by 305
  15. 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
  16. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 273
  17. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  18. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 114
  19. 2020Improving Image Captioning with Better Use of Captionscited by 84
  20. 2021Network Augmentation for Tiny Deep Learningcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 8; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  2. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  3. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 217
  4. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  5. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  6. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  7. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 47
  8. 2025Jet-Nemotron: Efficient Language Model with Post Neural Architecture Searchcited by 41
  9. 2025DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizercited by 13
  10. 2025LiT: Delving into a Simple Linear Diffusion Transformer for Image Generationcited by 10
  11. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 8
  12. 2024CASC: Condition-Aware Semantic Communication with Latent Diffusion Modelscited by 0
4 more
  1. 2025In the Blink of an Eye: Instant Game Map Editing using a Generative-AI Smart Brushcited by 0
  2. 2026JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Searchcited by 0
  3. 2026SmartFont: Dynamic Condition Allocation for Few-Shot Font Generationcited by 0
  4. 2026Space Generative AI with Solar Energy Harvestingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Simple Approach to Unifying Diffusion-based Conditional Generation
  2. 2024CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation
  3. 2025CPO: Condition Preference Optimization for Controllable Image Generation
  4. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation
  5. 2023Conditional Image Generation with Pretrained Generative Model
  6. 2022On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models
  7. 2024DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation
  8. 2023Conditional Generation from Unconditional Diffusion Models using Denoiser Representations