mlsanityData to 7 Oct 2026

AI & Human Co-Improvement for Safer Co-Superintelligence

arXiv 2512.05356 ↗First posted 5 Dec 2025cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Self-improvement is a goal currently exciting the field of AI, but is fraught with danger, and may take time to fully achieve. We advocate that a more achievable and better goal for humanity is to maximize co-improvement: collaboration between human researchers and AIs to achieve co-superintelligence. That is, specifically targeting improving AI systems' ability to work with human researchers to conduct AI research together, from ideation to experimentation, in order to both accelerate AI research and to generally endow both AIs and humans with safer superintelligence through their symbiosis. Focusing on including human research improvement in the loop will both get us there faster, and more safely.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 43 of its 66 references that are in the map. First posted on arXiv 5 Dec 2025.

Builds on

43 of 66 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2017Deep reinforcement learning from human preferencescited by 3,846
  11. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
31 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  7. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  8. 2024Self-Rewarding Language Modelscited by 654
  9. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  10. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  11. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  12. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  13. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  14. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  15. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  16. 2024Iterative Reasoning Preference Optimizationcited by 244
  17. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  18. 2023An Overview of Catastrophic AI Riskscited by 214
  19. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 212
  20. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  21. 2018The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigationcited by 159
  22. 2020Learning to Complement Humanscited by 123
  23. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 111
  24. 2019AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligencecited by 84
  25. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  26. 2024Self-Taught Evaluatorscited by 70
  27. 2025Self-Challenging Language Model Agentscited by 56
  28. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 30
  29. 2023Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programmingcited by 20
  30. 2025What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversitycited by 7
  31. 2024"Should I Give Up Now?" Investigating LLM Pitfalls in Software Engineeringcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
  2. 2026The Red Queen G"odel Machine: Co-Evolving Agents and Their Evaluatorscited by 16
  3. 2026On Safety Risks in Experience-Driven Self-Evolving Agentscited by 12
  4. 2026Autodata: An agentic data scientist to create high quality synthetic datacited by 12
  5. 2026Learning to Simulate Human Dialoguecited by 9
  6. 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learningcited by 7
  7. 2026Humans are Missing from AI Coding Agent Researchcited by 3
  8. 2026Recursive self-improvement of AI research agentscited by 3