A Black Swan Hypothesis: The Role of Human Irrationality in AI Safety
Not on the map. Why?
Abstract (arXiv)
Black swan events are statistically rare occurrences that carry extremely high risks. A typical view of defining black swan events is heavily assumed to originate from an unpredictable time-varying environments; however, the community lacks a comprehensive definition of black swan events. To this end, this paper challenges that the standard view is incomplete and claims that high-risk, statistically rare events can also occur in unchanging environments due to human misperception of their value and likelihood, which we call as spatial black swan event. We first carefully categorize black swan events, focusing on spatial black swan events, and mathematically formalize the definition of black swan events. We hope these definitions can pave the way for the development of algorithms to prevent such events by rationally correcting human perception.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 93 references that are in the map. First posted on arXiv 25 Jul 2024.
Builds on
18 of 93 references.
References · 18
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
- 2017A Distributional Perspective on Reinforcement Learningcited by 743
- 2019AutoML: A Survey of the State-of-the-Artcited by 388
- 2021Trustworthy AI: From Principles to Practicescited by 206
- 2013Risk-sensitive Reinforcement Learningcited by 129
- 2019Model-Based Reinforcement Learning with a Generative Model is Minimax Optimalcited by 124
- 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 71
- 2015Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Controlcited by 34
- 2022Settling the Reward Hypothesiscited by 33
- 2021Regret Minimization in Heavy-Tailed Banditscited by 24
- 2018Near-Optimal Time and Sample Complexities for Solving Discounted Markov Decision Process with a Generative Modelcited by 22
6 more
- 2019On Value Functions and the Agent-Environment Boundarycited by 19
- 2022The Quest for a Common Model of the Intelligent Decision Makercited by 16
- 2022Utility Theory for Sequential Decision Makingcited by 8
- 2023Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewardscited by 8
- 2023Conditions on Preference Relations that Guarantee the Existence of Optimal Policiescited by 3
- 2024Preparing for Black Swans: The Antifragility Imperative for Machine Learningcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.