mlsanityData to 7 Oct 2026

Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation

arXiv 2607.08758 ↗First posted 9 Jul 2026cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 53 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

42 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025AI-Researcher: Autonomous Scientific Innovation24 May

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  12. 2016YOLO9000: Better, Faster, Strongercited by 1,429
29 more
  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  5. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  6. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  9. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  10. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  11. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  12. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  13. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
  14. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 174
  15. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  16. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
  17. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  18. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 112
  19. 2020Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articlescited by 96
  20. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 94
  21. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60
  22. 2024Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideascited by 52
  23. 2024LitSearch: A Retrieval Benchmark for Scientific Literature Searchcited by 47
  24. 2025Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflowscited by 25
  25. 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
  26. 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 13
  27. 2025Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Networkcited by 5
  28. 2026Intern-Atlas: A Methodological Evolution Graph as Research Infrastructure for AI Scientistscited by 5
  29. 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
  2. 2026NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?
  3. 2025CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
  4. 2026LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
  5. 2025Idea2Plan: Exploring AI-Powered Research Planning
  6. 2026AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
  7. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Research
  8. 2026BABE: Biology Arena BEnchmark