Supertrust foundational alignment: mutual trust must replace permanent control for safe superintelligence
Not on the map. Why?
Abstract (arXiv)
It's widely expected that humanity will someday create AI systems vastly more intelligent than us, leading to the unsolved alignment problem of "how to control superintelligence." However, this commonly expressed problem is not only self-contradictory and likely unsolvable, but current strategies to ensure permanent control effectively guarantee that superintelligent AI will distrust humanity and consider us a threat. Such dangerous representations, already embedded in current models, will inevitably lead to an adversarial relationship and may even trigger the extinction event many fear. As AI leaders continue to "raise the alarm" about uncontrollable AI, further embedding concerns about it "getting out of our control" or "going rogue," we're unintentionally reinforcing our threat and deepening the risks we face. The rational path forward is to strategically replace intended permanent control with intrinsic mutual trust at the foundational level. The proposed Supertrust alignment meta-strategy seeks to accomplish this by modeling instinctive familial trust, representing superintelligence as the evolutionary child of human intelligence, and implementing temporary controls/constraints in the manner of effective parenting. Essentially, we're creating a superintelligent "child" that will be exponentially smarter and eventually independent of our control. We therefore have a critical choice: continue our controlling intentions and usher in a brief period of dominance followed by extreme hardship for humanity, or intentionally create the foundational mutual trust required for long-term safe coexistence.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 56 references that are in the map. First posted on arXiv 29 Jul 2024.
Builds on
12 of 56 references.
References · 12
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023A Survey on Multimodal Large Language Modelscited by 898
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
- 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
- 2023AI Alignment: A Comprehensive Surveycited by 256
- 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
- 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
- 2023Specific versus General Principles for Constitutional AIcited by 41
- 2024Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalizationcited by 23
- 2024A Moral Imperative: The Need for Continual Superalignment of Large Language Modelscited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.