# Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers

arXiv: https://arxiv.org/abs/2311.10242 · HTML: https://mlsanity.com/paper/2311.10242/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 76 of its 256 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 76)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)

## Built on by (most cited first, 30 of 36)

- [SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding](https://mlsanity.com/paper/2505.16630/) (2025)
- [Leveraging Generative AI for Urban Digital Twins: A Scoping Review on the Autonomous Generation of Urban Data, Scenarios, Designs, and 3D City Models for Smart City Advancement](https://mlsanity.com/paper/2405.19464/) (2024)
- [DropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation](https://mlsanity.com/paper/2503.06053/) (2025)
- [Generative AI in Industrial Machine Vision -- A Review](https://mlsanity.com/paper/2408.10775/) (2024)
- [A theoretical framework for overfitting in energy-based modeling](https://mlsanity.com/paper/2501.19158/) (2025)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications](https://mlsanity.com/paper/2510.21887/) (2025)
- [Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models](https://mlsanity.com/paper/2506.06018/) (2025)
- [Generative AI for Testing of Autonomous Driving Systems: A Survey](https://mlsanity.com/paper/2508.19882/) (2025)
- [Multimodal Human-Autonomous Agents Interaction Using Pre-Trained Language and Visual Foundation Models](https://mlsanity.com/paper/2403.12273/) (2024)
- [Aligning Generalisation Between Humans and Machines](https://mlsanity.com/paper/2411.15626/) (2024)
- [A Decade of Deep Learning for Remote Sensing Spatiotemporal Fusion: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2504.00901/) (2025)
- [Taxonomy of migration scenarios for Qiskit refactoring using LLMs](https://mlsanity.com/paper/2506.07135/) (2025)
- [Artificial Intelligence for Atmospheric Sciences: A Research Roadmap](https://mlsanity.com/paper/2506.16281/) (2025)
- [A Comprehensive Survey on Network Traffic Synthesis: From Statistical Models to Deep Learning](https://mlsanity.com/paper/2507.01976/) (2025)
- [Ethical Concerns of Generative AI and Mitigation Strategies: A Systematic Mapping Study](https://mlsanity.com/paper/2502.00015/) (2025)
- [Generative AI in Science: Applications, Challenges, and Emerging Questions](https://mlsanity.com/paper/2507.08310/) (2025)
- [GenUQ: Predictive Uncertainty Estimates via Generative Hyper-Networks](https://mlsanity.com/paper/2509.21605/) (2025)
- [A parameter-efficient three-branch architecture for multimodal misinformation detection with limited annotations](https://mlsanity.com/paper/2512.20257/) (2025)
- [NijiGAN: Transform What You See into Anime with Contrastive Semi-Supervised Learning and Neural Ordinary Differential Equations](https://mlsanity.com/paper/2412.19455/) (2024)
- [Comparison of Autoencoders for tokenization of ASL datasets](https://mlsanity.com/paper/2501.06942/) (2025)
- [Datum-wise Transformer for Synthetic Tabular Data Detection in the Wild](https://mlsanity.com/paper/2504.08829/) (2025)
- [Multimodal Cinematic Video Synthesis Using Text-to-Image and Audio Generation Models](https://mlsanity.com/paper/2506.10005/) (2025)
- [Mechanistic Interpretability in the Presence of Architectural Obfuscation](https://mlsanity.com/paper/2506.18053/) (2025)
- [Robust Detection of Synthetic Tabular Data under Schema Variability](https://mlsanity.com/paper/2509.00092/) (2025)
- [K-Syn: K-space Data Synthesis in Ultra Low-data Regimes](https://mlsanity.com/paper/2509.06997/) (2025)
- [Provenance of AI-Generated Images: A Vector Similarity and Blockchain-based Approach](https://mlsanity.com/paper/2510.17854/) (2025)
- [Batch Transformer Architecture: Case of Synthetic Image Generation for Emotion Expression Facial Recognition](https://mlsanity.com/paper/2511.11754/) (2025)
- [Evasion of IoT Malware Detection via Dummy Code Injection](https://mlsanity.com/paper/2602.08170/) (2026)
- [Mapping Recent Shifts in Digital Art via Conference Discourse: AI, XR, the Metaverse, and Blockchain/NFTs (2021-2025)](https://mlsanity.com/paper/2604.16360/) (2026)

## Abstract (arXiv)

The launch of ChatGPT has garnered global attention, marking a significant milestone in the field of Generative Artificial Intelligence. While Generative AI has been in effect for the past decade, the introduction of ChatGPT has ignited a new wave of research and innovation in the AI domain. This surge in interest has led to the development and release of numerous cutting-edge tools, such as Bard, Stable Diffusion, DALL-E, Make-A-Video, Runway ML, and Jukebox, among others. These tools exhibit remarkable capabilities, encompassing tasks ranging from text generation and music composition, image creation, video production, code generation, and even scientific work. They are built upon various state-of-the-art models, including Stable Diffusion, transformer models like GPT-3 (recent GPT-4), variational autoencoders, and generative adversarial networks. This advancement in Generative AI presents a wealth of exciting opportunities and, simultaneously, unprecedented challenges. Throughout this paper, we have explored these state-of-the-art models, the diverse array of tasks they can accomplish, the challenges they pose, and the promising future of Generative Artificial Intelligence.