The Variance Paradox: How AI Reduces Diversity but Increases Novelty
Not on the map. Why?
Abstract (arXiv)
The diversity of human expression is the raw material of discovery. Generative artificial intelligence threatens this resource even as it promises to accelerate innovation, a paradox now visible across science, culture, and professional work. We propose a framework to explain this tension. AI systems compress informational variance through statistical optimization, and users amplify this effect through epistemic deference. We call this process the AI Prism. Yet this same compression can enable novelty. Standardized forms travel across domain boundaries, lowering translation costs and creating opportunities for recombination that we term the Paradoxical Bridge. The interaction produces a U-shaped temporal dynamic, an initial decline in diversity followed by recombinant innovation, but only when humans actively curate rather than passively defer. The framework generates testable predictions about when compression constrains versus amplifies creativity. As AI becomes infrastructure for knowledge work, managing this dynamic is essential. Without intervention, the conditions for recovery may not arrive.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 43 of its 172 references that are in the map. First posted on arXiv 20 Aug 2025.
Builds on
43 of 172 references.
References · 43
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
- 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
- 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 167
- 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
- 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
- 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
- 2020Extractive Opinion Summarization in Quantized Transformer Spacescited by 76
- 2023Machine Culturecited by 55
- 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
31 more
- 2023Opportunities and Risks of LLMs for Scalable Deliberation with Poliscited by 42
- 2024Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluationcited by 41
- 2022What is it like to program with artificial intelligence?cited by 39
- 2020OpinionDigest: A Simple Framework for Opinion Summarizationcited by 37
- 2023CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AIcited by 34
- 2025The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Modelscited by 29
- 2023Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Noveltycited by 27
- 2024Generative Monoculture in Large Language Modelscited by 22
- 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 21
- 2024LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Researchcited by 17
- 2023Exploring Perspectives on the Impact of Artificial Intelligence on the Creativity of Knowledge Work: Beyond Mechanised Plagiarism and Stochastic Parrotscited by 11
- 2025The Lock-in Hypothesis: Stagnation by Algorithmcited by 11
- 2023Human-AI Interactions and Societal Pitfallscited by 9
- 2023Capturing Humans' Mental Models of AI: An Item Response Theory Approachcited by 8
- 2024Epistemic Integrity in Large Language Modelscited by 7
- 2024Steering AI-Driven Personalization of Scientific Text for General Audiencescited by 7
- 2025Variance reduction in output from generative AIcited by 6
- 2025The Dynamics of Collective Creativity in Human-AI Hybrid Societiescited by 5
- 2025How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responsescited by 5
- 2018Combinets: Creativity via Recombination of Neural Networkscited by 4
- 2024AI as Extraherics: Fostering Higher-order Thinking Skills in Human-AI Interactioncited by 4
- 2025Human Misperception of Generative-AI Alignment: A Laboratory Experimentcited by 4
- 2025Aspirational Affordances of AIcited by 4
- 2025System 0: Transforming Artificial Intelligence into a Cognitive Extensioncited by 3
- 2023The Innovation Paradox: Concept Space Expansion with Diminishing Originality and the Promise of Creative AIcited by 2
- 2023Quantifying the Benefit of Artificial Intelligence for Scientific Researchcited by 2
- 2023Latent Lab: Large Language Models for Knowledge Explorationcited by 2
- 2024LLM2TEA: An Agentic AI Designer for Discovery with Generative Evolutionary Multitaskingcited by 2
- 2025Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Workcited by 2
- 2024Extracting effective solutions hidden in large language models via generated comprehensive specialists: case studies in developing electronic devicescited by 1
- 2025IDRBench: Understanding the Capability of Large Language Models on Interdisciplinary Researchcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.