mlsanityData to 14 Oct 2026

The Singapore Consensus on Global AI Safety Research Priorities

arXiv 2506.20702 ↗First posted 25 Jun 2025cs.AI cs.CYcited by 10

Not on the map. Why?

Abstract (arXiv)

Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to ensure that AI is safe, i.e., trustworthy, reliable, and secure. Building a trusted ecosystem is therefore essential -- it helps people embrace AI with confidence and gives maximal space for innovation while avoiding backlash. The "2025 Singapore Conference on AI (SCAI): International Scientific Exchange on AI Safety" aimed to support research in this space by bringing together AI scientists across geographies to identify and synthesise research priorities in AI safety. This resulting report builds on the International AI Safety Report chaired by Yoshua Bengio and backed by 33 governments. By adopting a defence-in-depth model, this report organises AI safety research domains into three types: challenges with creating trustworthy AI systems (Development), challenges with evaluating their risks (Assessment), and challenges with monitoring and intervening after deployment (Control).

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 93 of its 140 references that are in the map. First posted on arXiv 25 Jun 2025.

Builds on

93 of 140 references.

References · 93

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  3. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2021Ethical and social risks of harm from Language Modelscited by 966
  7. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  8. 2023A Watermark for Large Language Modelscited by 725
  9. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  10. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  11. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  12. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
38 more
  1. 2018AI safety via debatecited by 400
  2. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  3. 2024Alignment faking in large language modelscited by 360
  4. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  5. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  6. 2016Cooperative Inverse Reinforcement Learningcited by 300
  7. 2023AI Alignment: A Comprehensive Surveycited by 263
  8. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 260
  9. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  10. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  11. 2021Trustworthy AI: From Principles to Practicescited by 207
  12. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  13. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  14. 2024A Roadmap to Pluralistic Alignmentcited by 184
  15. 2025Open Problems in Mechanistic Interpretabilitycited by 179
  16. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  17. 2023Model evaluation for extreme riskscited by 177
  18. 2025Multi-Agent Risks from Advanced AIcited by 163
  19. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  20. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
  21. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  22. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  23. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 126
  24. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  25. 2024Language Models Learn to Mislead Humans via RLHFcited by 124
  26. 2021Truthful AI: Developing and governing AI that does not liecited by 113
  27. 2024The Ethics of Advanced AI Assistantscited by 107
  28. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 105
  29. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 96
  30. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 92
  31. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  32. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  33. 2025Auditing language models for hidden objectivescited by 83
  34. 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
  35. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  36. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 72
  37. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 70
  38. 2025International AI Safety Reportcited by 67

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 4.

10 papers.

  1. 2025OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detectioncited by 10
  2. 2025Toward a Safe Internet of Agentscited by 6
  3. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2
  4. 2025Responsible AI Technical Reportcited by 1
  5. 2025From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrailscited by 1
  6. 2026Comprehensive AI governance requires addressing non-model gainscited by 1
  7. 2025Failure Modes of Maximum Entropy RLHFcited by 0
  8. 2026Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approachcited by 0
  9. 2026An Instrument to Evaluate Governance Proposals: AI Policy Analysis at Scalecited by 0
  10. 2026Agency as a New Organizing Abstraction for Computer Sciencecited by 0