Epistemic Norms for AI Safety and Alignment Research
Not on the map. Why?
Abstract (arXiv)
Mainstream AI research emphasises capability growth and tolerates low failure rates when average-case performance is high. AI safety and alignment research has a different mission: to ensure that catastrophic failures never occur, under sparse evidence, adversarial dynamics, and fat-tailed risk. We argue that the two domains differ along two analytically independent axes---it capability profile, demonstrating the absence of hazardous behaviours rather than the presence of positive capabilities, and it risk profile, bounding worst-case outcomes under fat-tailed uncertainty rather than optimising average-case performance---and that mainstream epistemic practices are inadequate on both. Building on a structured synthesis grounded in a preregistered bibliometric baseline, we identify five cross-cutting gap dimensions in current alignment research, including the near-absence of institutionalised independent verification. To address these gaps, we propose sc ECAISA, an Epistemic Code for AI Safety and Alignment comprising eight principles, a three-level scoring rubric, a four-level disclosure ladder that reconciles transparency with information-hazard and commercial-confidentiality constraints, a tiered applicability scheme, an information-hazard adjudication procedure, and seven anti-gaming mechanisms. sc ECAISA does not certify that any AI system is safe; it constrains how safety-relevant research claims are documented, checked, and relied upon, with auditability rather than certification as its governance target.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 39 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
8 of 39 references.
References · 8
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2020Learning to summarize from human feedbackcited by 2,562
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 102
- 2019A Step Toward Quantifying Independently Reproducible Machine Learning Researchcited by 77
- 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 9
- 2024Assurance of AI Systems From a Dependability Perspectivecited by 5
Built on by
Not cited yet.