mlsanityData to 7 Oct 2026

SciMON: Scientific Inspiration Machines Optimized for Novelty

arXiv 2305.14259 ↗First posted 23 May 2023cs.CL cs.AI cs.LGcited by 174

Not on the map. Why?

Cited by 38 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature. Work on literature-based hypothesis generation has traditionally focused on binary link prediction--severely limiting the expressivity of hypotheses. This line of work also does not focus on optimizing novelty. We take a dramatic departure with a novel setting in which models use as input background contexts (e.g., problems, experimental settings, goals), and output natural language ideas grounded in literature. We present SciMON, a modeling framework that uses retrieval of "inspirations" from past scientific papers, and explicitly optimizes for novelty by iteratively comparing to prior papers and updating idea suggestions until sufficient novelty is achieved. Comprehensive evaluations reveal that GPT-4 tends to generate ideas with overall low technical depth and novelty, while our methods partially mitigate this issue. Our work represents a first step toward evaluating and developing language models that generate new ideas derived from the scientific literature

Key facts. Cited by 174 papers in the map (data through 7 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

24 of 54 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  10. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
  11. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
  12. 2019Episodic Memory in Lifelong Language Learningcited by 188
12 more
  1. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  2. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  3. 2019Pretrained Language Models for Sequential Sentence Classificationcited by 53
  4. 2021MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NERcited by 42
  5. 2019PaperRobot: Incremental Draft Generation of Scientific Ideascited by 37
  6. 2021Learning When to Translate for Streaming Speechcited by 30
  7. 2021Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Studycited by 23
  8. 2022A Computational Inflection for Scientific Discoverycited by 22
  9. 2020AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approachcited by 21
  10. 2021SciCo: Hierarchical Cross-Document Coreference for Scientific Conceptscited by 14
  11. 2022tmVar 3.0: an improved variant concept recognition and normalization toolcited by 10
  12. 2021Context-Aware Interaction Network for Question Matchingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 33; 2025: 71; 2026: 66.

174 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  3. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  4. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 117
  5. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  6. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  7. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  8. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 94
  9. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  10. 2024Hypothesis Generation with Large Language Modelscited by 68
  11. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 68
  12. 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agentscited by 64
38 more
  1. 2025PaSa: An LLM Agent for Comprehensive Academic Paper Searchcited by 61
  2. 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 61
  3. 2025ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decompositioncited by 57
  4. 2024Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideascited by 52
  5. 2024Autonomous LLM-driven research from data to human-verifiable research paperscited by 50
  6. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
  7. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  8. 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challengescited by 41
  9. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  10. 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 40
  11. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  12. 2024Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluationcited by 39
  13. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  14. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 34
  15. 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 29
  16. 2025AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprisecited by 28
  17. 2024What Can Natural Language Processing Do for Peer Review?cited by 24
  18. 2024Can Large Language Models Unlock Novel Scientific Research Ideas?cited by 24
  19. 2025Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedbackcited by 23
  20. 2024SciPIP: An LLM-based Scientific Paper Idea Proposercited by 22
  21. 2025Sparks of Science: Hypothesis Generation Using Structured Paper Datacited by 22
  22. 2024Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaderscited by 21
  23. 2024Improving Scientific Hypothesis Generation with Knowledge Grounded Large Language Modelscited by 21
  24. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 20
  25. 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrationscited by 20
  26. 2024Literature Meets Data: A Synergistic Approach to Hypothesis Generationcited by 19
  27. 2026Towards Execution-Grounded Automated AI Researchcited by 19
  28. 2025RAG-Enhanced Collaborative LLM Agents for Drug Discoverycited by 18
  29. 2025All That Glitters is Not Novel: Plagiarism in AI Generated Researchcited by 17
  30. 2025SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Taskscited by 17
  31. 2024PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideationcited by 16
  32. 2024LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Researchcited by 16
  33. 2025IRIS: Interactive Research Ideation System for Accelerating Scientific Discoverycited by 16
  34. 2025How Far Are AI Scientists from Changing the World?cited by 15
  35. 2025OmniScientist: Toward a Co-evolving Ecosystem of Human and AI Scientistscited by 15
  36. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 14
  37. 2025MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Searchcited by 13
  38. 2025PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priorscited by 13

50 most cited of 174 All 174 (JSON)