Sci-Reasoning: A Dataset Decoding AI Innovation Patterns
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
Abstract (arXiv)
While AI innovation accelerates rapidly, the intellectual process behind breakthroughs -- how researchers identify gaps, synthesize prior work, and generate insights -- remains poorly understood. The lack of structured data on scientific reasoning hinders systematic analysis and development of AI research agents. We introduce Sci-Reasoning, the first dataset capturing the intellectual synthesis behind high-quality AI research. Using community-validated quality signals and an LLM-accelerated, human-verified pipeline, we trace Oral and Spotlight papers across NeurIPS, ICML, and ICLR (2023-2025) to its key predecessors, articulating specific reasoning links in a structured format. Our analysis identifies 15 distinct thinking patterns, with three dominant strategies accounting for 52.7%: Gap-Driven Reframing (24.2%), Cross-Domain Synthesis (18.0%), and Representation Shift (10.5%). The most powerful innovation recipes combine multiple patterns: Gap-Driven Reframing + Representation Shift, Cross-Domain Synthesis + Representation Shift, and Gap-Driven Reframing + Cross-Domain Synthesis. This dataset enables quantitative studies of scientific progress and provides structured reasoning trajectories for training the next generation AI research agents.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 31 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
17 of 31 references.
References · 17
- 2017Attention Is All You Needcited by 59,932
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 135
- 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
- 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
- 2024LitLLM: A Toolkit for Scientific Literature Reviewcited by 61
- 2024Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Servicescited by 44
- 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
- 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agentscited by 31
5 more
- 2023CitePrompt: Using Prompts to Identify Citation Intent in Scientific Paperscited by 12
- 2024The Nature of NLP: Analyzing Contributions in NLP Paperscited by 12
- 2024PST-Bench: Tracing and Benchmarking the Source of Publicationscited by 7
- 2025Research Knowledge Graphs: the Shifting Paradigm of Scholarly Information Representationcited by 2
- 2022See further upon the giants: Quantifying intellectual lineage in sciencecited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026SciAtlas: A Computable Atlas of Science for Knowledge-Grounded AI Research
- 2026AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
- 2025AI-Researcher: Autonomous Scientific Innovation
- 2026AI scientists produce results without reasoning scientifically
- 2026Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
- 2026Can AI Agents Synthesize Scientific Conclusions?
- 2026SciResearcher: Scaling Deep Research Agents for Frontier Scientific Reasoning
- 2024AI Knowledge and Reasoning: Emulating Expert Creativity in Scientific Research