# Art and the science of generative AI: A deeper dive

arXiv: https://arxiv.org/abs/2306.04141 · HTML: https://mlsanity.com/paper/2306.04141/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 141 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations](https://mlsanity.com/paper/2301.04246/) (2023)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Mode Regularized Generative Adversarial Networks](https://mlsanity.com/paper/1612.02136/) (2016)
- [Foundation Models and Fair Use](https://mlsanity.com/paper/2303.15715/) (2023)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting](https://mlsanity.com/paper/1906.09531/) (2019)
- [Adversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples](https://mlsanity.com/paper/2002.12749/) (2020)
- [Understanding and Creating Art with AI: Review and Outlook](https://mlsanity.com/paper/2102.09109/) (2021)
- [Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds](https://mlsanity.com/paper/2105.06496/) (2021)
- [Why AI is Harder Than We Think](https://mlsanity.com/paper/2104.12871/) (2021)
- [Studying Bias in GANs through the Lens of Race](https://mlsanity.com/paper/2209.02836/) (2022)
- [Can Computers Create Art?](https://mlsanity.com/paper/1801.04486/) (2018)
- [Active Divergence with Generative Deep Learning -- A Survey and Taxonomy](https://mlsanity.com/paper/2107.05599/) (2021)
- [Toward Modeling Creative Processes for Algorithmic Painting](https://mlsanity.com/paper/2205.01605/) (2022)
- [When happy accidents spark creativity: Bringing collaborative speculation to life with generative AI](https://mlsanity.com/paper/2206.00533/) (2022)

## Built on by (most cited first, 30 of 95)

- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference](https://mlsanity.com/paper/2310.03003/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Raising the Bar of AI-generated Image Detection with CLIP](https://mlsanity.com/paper/2312.00195/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI](https://mlsanity.com/paper/2310.16787/) (2023)
- [A Bias-Free Training Paradigm for More General AI-generated Image Detection](https://mlsanity.com/paper/2412.17671/) (2024)
- [Consent in Crisis: The Rapid Decline of the AI Data Commons](https://mlsanity.com/paper/2407.14933/) (2024)
- [D3: Scaling Up Deepfake Detection by Learning from Discrepancy](https://mlsanity.com/paper/2404.04584/) (2024)
- [GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models](https://mlsanity.com/paper/2310.00737/) (2023)
- [Image-GS: Content-Adaptive Image Representation via 2D Gaussians](https://mlsanity.com/paper/2407.01866/) (2024)
- [CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent](https://mlsanity.com/paper/2310.06155/) (2023)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Model-as-a-Service (MaaS): A Survey](https://mlsanity.com/paper/2311.05804/) (2023)
- [Data Attribution for Text-to-Image Models by Unlearning Synthesized Images](https://mlsanity.com/paper/2406.09408/) (2024)
- [Mapping the Ethics of Generative AI: A Comprehensive Scoping Review](https://mlsanity.com/paper/2402.08323/) (2024)
- [A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models](https://mlsanity.com/paper/2308.12143/) (2023)
- [GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs](https://mlsanity.com/paper/2502.11925/) (2025)
- [User Experience Design Professionals' Perceptions of Generative Artificial Intelligence](https://mlsanity.com/paper/2309.15237/) (2023)
- [Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?](https://mlsanity.com/paper/2404.12691/) (2024)
- [FlexOlmo: Open Language Models for Flexible Data Use](https://mlsanity.com/paper/2507.07024/) (2025)
- [Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images](https://mlsanity.com/paper/2502.11989/) (2025)
- [MLaGA: Multimodal Large Language and Graph Assistant](https://mlsanity.com/paper/2506.02568/) (2025)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Foundation Models for Education: Promises and Prospects](https://mlsanity.com/paper/2405.10959/) (2024)
- [Exploring the Use of Abusive Generative AI Models on Civitai](https://mlsanity.com/paper/2407.12876/) (2024)
- [Color Alignment in Diffusion](https://mlsanity.com/paper/2503.06746/) (2025)
- [AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detection](https://mlsanity.com/paper/2504.20865/) (2025)

## Abstract (arXiv)

A new class of tools, colloquially called generative AI, can produce high-quality artistic media for visual arts, concept art, music, fiction, literature, video, and animation. The generative capabilities of these tools are likely to fundamentally alter the creative processes by which creators formulate ideas and put them into production. As creativity is reimagined, so too may be many sectors of society. Understanding the impact of generative AI - and making policy decisions around it - requires new interdisciplinary scientific inquiry into culture, economics, law, algorithms, and the interaction of technology and creativity. We argue that generative AI is not the harbinger of art's demise, but rather is a new medium with its own distinct affordances. In this vein, we consider the impacts of this new medium on creators across four themes: aesthetics and culture, legal questions of ownership and credit, the future of creative work, and impacts on the contemporary media ecosystem. Across these themes, we highlight key research questions and directions to inform policy and beneficial uses of the technology.