# Natural Selection Favors AIs over Humans arXiv: https://arxiv.org/abs/2303.16200 ยท HTML: https://mlsanity.com/paper/2303.16200/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 11 of its 95 references that are in the map. - First posted on arXiv 28 Mar 2023. ## Builds on (most cited first, 11 of 11) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016) - [The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models](https://mlsanity.com/paper/2201.03544/) (2022) - [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021) - [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019) - [Is Power-Seeking AI an Existential Risk?](https://mlsanity.com/paper/2206.13353/) (2022) - [Forecasting Future World Events with Neural Networks](https://mlsanity.com/paper/2206.15474/) (2022) - [What Would Jiminy Cricket Do? Towards Agents That Behave Morally](https://mlsanity.com/paper/2110.13136/) (2021) - [Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications](https://mlsanity.com/paper/2001.10488/) (2020) ## Built on by (most cited first, 24 of 24) - [Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://mlsanity.com/paper/2306.04488/) (2023) - [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023) - [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023) - [WARM: On the Benefits of Weight Averaged Reward Models](https://mlsanity.com/paper/2401.12187/) (2024) - [Deception Abilities Emerged in Large Language Models](https://mlsanity.com/paper/2307.16513/) (2023) - [Visibility into AI Agents](https://mlsanity.com/paper/2401.13138/) (2024) - [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023) - [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025) - [Adversarial Attacks and Defenses in Large Language Models: Old and New Threats](https://mlsanity.com/paper/2310.19737/) (2023) - [Managing extreme AI risks amid rapid progress](https://mlsanity.com/paper/2310.17688/) (2023) - [WARP: On the Benefits of Weight Averaged Rewarded Policies](https://mlsanity.com/paper/2406.16768/) (2024) - [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023) - [Agent Alignment in Evolving Social Norms](https://mlsanity.com/paper/2401.04620/) (2024) - [Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective](https://mlsanity.com/paper/2504.03255/) (2025) - [Superintelligence Strategy: Expert Version](https://mlsanity.com/paper/2503.05628/) (2025) - [Towards social generative AI for education: theory, practices and ethics](https://mlsanity.com/paper/2306.10063/) (2023) - [Fairness in AI and Its Long-Term Implications on Society](https://mlsanity.com/paper/2304.09826/) (2023) - [On the Undecidability of Artificial Intelligence Alignment: Machines that Halt](https://mlsanity.com/paper/2408.08995/) (2024) - [The Digital Ecosystem of Beliefs: does evolution favour AI over humans?](https://mlsanity.com/paper/2412.14500/) (2024) - [Joint Scoring Rules: Zero-Sum Competition Avoids Performative Prediction](https://mlsanity.com/paper/2412.20732/) (2024) - [Eigenism: Ethics for a Human-AI Future](https://mlsanity.com/paper/2606.12420/) (2026) - [A mathematical theory of evolution for self-designing AIs](https://mlsanity.com/paper/2604.05142/) (2026) - [Artificial Id: Drive and Persistent Alignment in Agentic AI](https://mlsanity.com/paper/2609.11911/) (2026) - [Legislating World-Model-Based Planning with Legal Reasoning](https://mlsanity.com/paper/2609.15113/) (2026) ## Abstract (arXiv) For billions of years, evolution has been the driving force behind the development of life, including humans. Evolution endowed humans with high intelligence, which allowed us to become one of the most successful species on the planet. Today, humans aim to create artificial intelligence systems that surpass even our own intelligence. As artificial intelligences (AIs) evolve and eventually surpass us in all domains, how might evolution shape our relations with AIs? By analyzing the environment that is shaping the evolution of AIs, we argue that the most successful AI agents will likely have undesirable traits. Competitive pressures among corporations and militaries will give rise to AI agents that automate human roles, deceive others, and gain power. If such agents have intelligence that exceeds that of humans, this could lead to humanity losing control of its future. More abstractly, we argue that natural selection operates on systems that compete and vary, and that selfish species typically have an advantage over species that are altruistic to other species. This Darwinian logic could also apply to artificial agents, as agents may eventually be better able to persist into the future if they behave selfishly and pursue their own interests with little regard for humans, which could pose catastrophic risks. To counteract these risks and evolutionary forces, we consider interventions such as carefully designing AI agents' intrinsic motivations, introducing constraints on their actions, and institutions that encourage cooperation. These steps, or others that resolve the problems we pose, will be necessary in order to ensure the development of artificial intelligence is a positive one.