mlsanityData to 14 Oct 2026

Vision-centric generative AI models: A software-hardware perspective

arXiv 2608.27199 ↗First posted 27 Aug 2026cs.CV cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision generative artificial intelligence (AI) has emerged as one of the most rapidly advancing areas of deep learning. The explosion of multimodal models has made them widely associated with text-to-image applications running on large datacentres. However, vision generative models are equally needed in applications that operate under strict hardware constraints at the edge, including autonomous vehicles, agricultural sensors, and mobile devices. In this Perspective, we argue that progress in vision generative AI has been driven by output quality, with hardware evolving reactively to accommodate growing model demands. We quantify the parameter cost and energy efficiency of these models across a range of accelerator platforms, and map four generative model families against seven real-world application domains. Finally, we advocate a software-hardware co-design approach, where deployment constraints are considered from the start of the design process, ensuring that the "right model" runs on the "right hardware" to serve the "right application", making generative AI deployment sustainable and accessible across a much broader range of platforms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 68 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

27 of 68 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  10. 2013Auto-Encoding Variational Bayescited by 4,053
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  12. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
15 more
  1. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  2. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 202
  3. 2020The Hardware Lotterycited by 160
  4. 2022Wavelet Diffusion Models are fast and scalable Image Generatorscited by 117
  5. 2022Diffusion Probabilistic Models beat GANs on Medical Imagescited by 77
  6. 2024Autoregressive Models in Vision: A Surveycited by 39
  7. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  8. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 33
  9. 2025Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translationcited by 8
  10. 2024An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViTcited by 4
  11. 2024Latent Denoising Diffusion GAN: Faster sampling, Higher image qualitycited by 4
  12. 2023COLA: Characterizing and Optimizing the Tail Latency for Safe Level-4 Autonomous Vehicle Systemscited by 3
  13. 2025Leveraging Compute-in-Memory for Efficient Generative Model Inference in TPUscited by 3
  14. 2025SD-Acc: Accelerating Stable Diffusion through Phase-aware Sampling and Hardware Co-Optimizationscited by 2
  15. 2025Beyond Moore's Law: Harnessing the Redshift of Generative AI with Effective Hardware-Software Co-Designcited by 1

Built on by

Not cited yet.