mlsanityData to 14 Oct 2026

Dark Speculation: Combining Qualitative and Quantitative Understanding in Frontier AI Risk Analysis

arXiv 2511.21838 ↗First posted 26 Nov 2025cs.CY cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Estimating catastrophic harms from frontier AI is hindered by deep ambiguity: many of its risks are not only unobserved but unanticipated by analysts. The central limitation of current risk analysis is the inability to populate the textitcatastrophic event space, or the set of potential large-scale harms to which probabilities might be assigned. This intractability is worsened by the textitLucretius problem, or the tendency to infer future risks only from past experience. We propose a process of textitdark speculation, in which systematically generating and refining catastrophic scenarios ("qualitative" work) is coupled with estimating their likelihoods and associated damages (quantitative underwriting analysis). The idea is neither to predict the future nor to enable insurance for its own sake, but to use narrative and underwriting tools together to generate probability distributions over outcomes. We formalize this process using a simplified catastrophic L'evy stochastic framework and propose an iterative institutional design in which (1) speculation (including scenario planning) generates detailed catastrophic event narratives, (2) insurance underwriters assign probabilistic and financial parameters to these narratives, and (3) decision-makers synthesize the results into summary statistics to inform judgment. Analysis of the model reveals the value of (a) maintaining independence between speculation and underwriting, (b) analyzing multiple risk categories in parallel, and (c) generating "thick" catastrophic narrative rich in causal (counterfactual) and mitigative detail. While the approach cannot eliminate deep ambiguity, it offers a systematic approach to reason about extreme, low-probability events in frontier AI, tempering complacency and overreaction. The framework is adaptable for iterative use and can be further augmented with AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 47 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

9 of 47 references.

References · 9

  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  2. 2024Frontier Models are Capable of In-context Schemingcited by 287
  3. 2022Is Power-Seeking AI an Existential Risk?cited by 105
  4. 2016The Off-Switch Gamecited by 97
  5. 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 97
  6. 2025Gradual Disempowerment: Systemic Existential Risks from Incremental AI Developmentcited by 46
  7. 2024Random Tree Model of Meaningful Memorycited by 4
  8. 2024An FDA for AI? Pitfalls and Plausibility of Approval Regulation for Frontier Artificial Intelligencecited by 3
  9. 2024Strategic Insights from Simulation Gaming of AI Race Dynamicscited by 3

Built on by

Not cited yet.