# Vision-centric generative AI models: A software-hardware perspective

arXiv: https://arxiv.org/abs/2608.27199 · HTML: https://mlsanity.com/paper/2608.27199/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 68 references that are in the map.
- First posted on arXiv 27 Aug 2026.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis](https://mlsanity.com/paper/2206.09479/) (2022)
- [The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization](https://mlsanity.com/paper/2505.06371/) (2025)
- [Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translation](https://mlsanity.com/paper/2508.09177/) (2025)
- [An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViT](https://mlsanity.com/paper/2403.20230/) (2024)
- [Latent Denoising Diffusion GAN: Faster sampling, Higher image quality](https://mlsanity.com/paper/2406.11713/) (2024)
- [COLA: Characterizing and Optimizing the Tail Latency for Safe Level-4 Autonomous Vehicle Systems](https://mlsanity.com/paper/2305.07147/) (2023)
- [Leveraging Compute-in-Memory for Efficient Generative Model Inference in TPUs](https://mlsanity.com/paper/2503.00461/) (2025)
- [SD-Acc: Accelerating Stable Diffusion through Phase-aware Sampling and Hardware Co-Optimizations](https://mlsanity.com/paper/2507.01309/) (2025)
- [Beyond Moore's Law: Harnessing the Redshift of Generative AI with Effective Hardware-Software Co-Design](https://mlsanity.com/paper/2504.06531/) (2025)

## Abstract (arXiv)

Vision generative artificial intelligence (AI) has emerged as one of the most rapidly advancing areas of deep learning. The explosion of multimodal models has made them widely associated with text-to-image applications running on large datacentres. However, vision generative models are equally needed in applications that operate under strict hardware constraints at the edge, including autonomous vehicles, agricultural sensors, and mobile devices. In this Perspective, we argue that progress in vision generative AI has been driven by output quality, with hardware evolving reactively to accommodate growing model demands. We quantify the parameter cost and energy efficiency of these models across a range of accelerator platforms, and map four generative model families against seven real-world application domains. Finally, we advocate a software-hardware co-design approach, where deployment constraints are considered from the start of the design process, ensuring that the "right model" runs on the "right hardware" to serve the "right application", making generative AI deployment sustainable and accessible across a much broader range of platforms.