mlsanityData to 7 Oct 2026

On the Challenges and Opportunities in Generative AI

arXiv 2403.00025 ↗First posted 28 Feb 2024cs.LG cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervised learning paradigms, recent large-scale generative models show tremendous promise in synthesizing high-resolution images and text, as well as structured data such as videos and molecules. However, we argue that current large-scale generative AI models exhibit several fundamental shortcomings that hinder their widespread adoption across domains. In this work, our objective is to identify these issues and highlight key unresolved challenges in modern generative AI paradigms that should be addressed to further enhance their capabilities, versatility, and reliability. By identifying these challenges, we aim to provide researchers with insights for exploring fruitful research directions, thus fostering the development of more robust and accessible generative AI solutions.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 295 of its 391 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

295 of 391 references.

References · 295

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  8. 2023GPT-4 Technical Reportcited by 18,132
  9. 2022Training language models to follow instructions with human feedbackcited by 15,712
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2020Denoising Diffusion Implicit Modelscited by 9,655
38 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  13. 2017On Calibration of Modern Neural Networkscited by 4,860
  14. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  15. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  16. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  17. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  18. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  19. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  20. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  21. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  22. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  23. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  24. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  25. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  26. 2016Improved Techniques for Training GANscited by 2,938
  27. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  28. 2022Emergent Abilities of Large Language Modelscited by 2,483
  29. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  30. 2022Video Diffusion Modelscited by 2,280
  31. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  32. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  33. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  34. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  35. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  36. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  37. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  38. 2023Consistency Modelscited by 1,778

50 most cited of 295 All 295 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 16; 2026: 4.

31 papers.

Sort:

By area:

  1. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  2. 2024I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Tokencited by 26
  3. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  4. 2024Fast Samplers for Inverse Problems in Iterative Refinement Modelscited by 17
  5. 2024Unity by Diversity: Improved Representation Learning in Multimodal VAEscited by 13
  6. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  7. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
  8. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspectivecited by 7
  9. 2024Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughtscited by 5
  10. 2025Statistical Methods in Generative AIcited by 4
  11. 2025Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluationscited by 4
  12. 2025Beyond N-Grams: Rethinking Evaluation Metrics and Strategies for Multilingual Abstractive Summarizationcited by 3
19 more
  1. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  2. 2024SetPINNs: Set-based Physics-informed Neural Networkscited by 2
  3. 2024Rendering-Refined Stable Diffusion for Privacy Compliant Synthetic Datacited by 2
  4. 2025Contextual Memory Intelligence -- A Foundational Paradigm for Human-AI Collaboration and Reflective Generative AI Systemscited by 2
  5. 2024Tethering Broken Themes: Aligning Neural Topic Models with Labels and Authorscited by 1
  6. 2025Pretrained LLMs Learn Multiple Types of Uncertaintycited by 1
  7. 2025Generative AI as a Pillar for Predicting 2D and 3D Wildfire Spread: Beyond Physics-Based Models and Traditional Deep Learningcited by 1
  8. 2025A Distributed Generative AI Approach for Heterogeneous Multi-Domain Environments under Data Sharing constraintscited by 1
  9. 2026From Reflection to Repair: A Scoping Review of Dataset Documentation Toolscited by 1
  10. 2024Generative Topological Networkscited by 0
  11. 2024Progressive Monitoring of Generative Model Training Evolutioncited by 0
  12. 2025AI-Generated Content in Cross-Domain Applications: Research Trends, Challenges and Propositionscited by 0
  13. 2025Match Chat: Real Time Generative AI and Generative Computing for Tenniscited by 0
  14. 2025seqme: a Python library for evaluating biological sequence designcited by 0
  15. 2025Low-Bit, High-Fidelity: Optimal Transport Quantization for Flow Matchingcited by 0
  16. 2025OTARo: Once Tuning for All Precisions toward Robust On-Device LLMscited by 0
  17. 2026The Annotation Scarcity Paradox in Low-Resource NLP Evaluation: A Decade of Acceleration and Emerging Constraintscited by 0
  18. 2026Physics-informed Conditional Normalizing Flows for Angles-only Cislunar Orbit Determinationcited by 0
  19. 2026Adaptive Diffusion Freezing: Privacy-preserving Diffusion Models Against Membership Inference Attackscited by 0