# Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluation

arXiv: https://arxiv.org/abs/2409.14634 · HTML: https://mlsanity.com/paper/2409.14634/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 73 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 29 of 29)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation](https://mlsanity.com/paper/2402.09267/) (2024)
- [Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references](https://mlsanity.com/paper/1402.4578/) (2014)
- [Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity](https://mlsanity.com/paper/2111.08366/) (2021)
- [CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI](https://mlsanity.com/paper/2312.11949/) (2023)
- [LLMFactor: Extracting Profitable Factors through Prompts for Explainable Stock Movement Prediction](https://mlsanity.com/paper/2406.10811/) (2024)
- [Accelerating Innovation Through Analogy Mining](https://mlsanity.com/paper/1706.05585/) (2017)
- [A Computational Inflection for Scientific Discovery](https://mlsanity.com/paper/2205.02007/) (2022)
- [Augmenting Scientific Creativity with an Analogical Search Engine](https://mlsanity.com/paper/2205.15476/) (2022)
- [Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets](https://mlsanity.com/paper/2310.09985/) (2023)
- [Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability](https://mlsanity.com/paper/2401.08574/) (2024)
- [Ranked List Truncation for Large Language Model-based Re-Ranking](https://mlsanity.com/paper/2404.18185/) (2024)
- [TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents](https://mlsanity.com/paper/2312.01279/) (2023)
- [ReviewFlow: Intelligent Scaffolding to Support Academic Peer Reviewing](https://mlsanity.com/paper/2402.03530/) (2024)
- [DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries](https://mlsanity.com/paper/2310.04678/) (2023)
- [TWOLAR: a TWO-step LLM-Augmented distillation method for passage Reranking](https://mlsanity.com/paper/2403.17759/) (2024)
- [Scaling Creative Inspiration with Fine-Grained Functional Aspects of Ideas](https://mlsanity.com/paper/2102.09761/) (2021)
- [Digital Socrates: Evaluating LLMs through Explanation Critiques](https://mlsanity.com/paper/2311.09613/) (2023)
- [BioSpark: An End-to-End Generative System for Biological-Analogical Inspirations and Ideation](https://mlsanity.com/paper/2312.11388/) (2023)
- [Algorithmic Ways of Seeing: Using Object Detection to Facilitate Art Exploration](https://mlsanity.com/paper/2403.19174/) (2024)
- [VIVID: Human-AI Collaborative Authoring of Vicarious Dialogues from Lecture Videos](https://mlsanity.com/paper/2403.09168/) (2024)

## Built on by (most cited first, 30 of 41)

- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2505.13259/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback](https://mlsanity.com/paper/2410.04025/) (2024)
- [Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?](https://mlsanity.com/paper/2502.14297/) (2025)
- [Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context](https://mlsanity.com/paper/2412.17596/) (2024)
- [ResearchGym: Evaluating Language Model Agents on Real-World AI Research](https://mlsanity.com/paper/2602.15112/) (2026)
- [AI Scientists Fail Without Strong Implementation Capability](https://mlsanity.com/paper/2506.01372/) (2025)
- [IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery](https://mlsanity.com/paper/2504.16728/) (2025)
- [Literature-Grounded Novelty Assessment of Scientific Ideas](https://mlsanity.com/paper/2506.22026/) (2025)
- [CellForge: Agentic Design of Virtual Cell Models](https://mlsanity.com/paper/2508.02276/) (2025)
- [Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback](https://mlsanity.com/paper/2508.10795/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models](https://mlsanity.com/paper/2505.14599/) (2025)
- [Matter-of-Fact: A Benchmark for Verifying the Feasibility of Literature-Supported Claims in Materials Science](https://mlsanity.com/paper/2506.04410/) (2025)
- [TinyScientist: An Interactive, Extensible, and Controllable Framework for Building Research Agents](https://mlsanity.com/paper/2510.06579/) (2025)
- [MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation](https://mlsanity.com/paper/2509.21978/) (2025)
- [Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation](https://mlsanity.com/paper/2509.20553/) (2025)
- [Navigating Ideation Space: Decomposed Conceptual Representations for Positioning Scientific Ideas](https://mlsanity.com/paper/2601.08901/) (2026)
- [FutureGen: A RAG-based Approach to Generate the Future Work of Scientific Article](https://mlsanity.com/paper/2503.16561/) (2025)
- [From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Tasks](https://mlsanity.com/paper/2504.02780/) (2025)
- [Agent Ideate: A Framework for Product Idea Generation from Patents Using Agentic AI](https://mlsanity.com/paper/2507.01717/) (2025)
- [The Ramon Llull's Thinking Machine for Automated Ideation](https://mlsanity.com/paper/2508.19200/) (2025)
- [Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers](https://mlsanity.com/paper/2510.00361/) (2025)
- [COMPOSE: Composing Future Theorems from Citations and Formal Structure](https://mlsanity.com/paper/2605.30333/) (2026)
- [Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Work](https://mlsanity.com/paper/2503.18471/) (2025)
- [ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes](https://mlsanity.com/paper/2607.04439/) (2026)
- [Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation](https://mlsanity.com/paper/2608.01666/) (2026)
- [What Proves You Wrong: Benchmarking Language Models on Falsifiable Research Ideation](https://mlsanity.com/paper/2608.22948/) (2026)
- [Towards Automated Scoping of AI for Social Good Projects](https://mlsanity.com/paper/2504.20010/) (2025)

## Abstract (arXiv)

The scientific ideation process often involves blending facets of existing papers to create new ideas. We contribute Scideator, the first human-LLM system for facet-based scientific ideation. Starting from user-provided papers, Scideator extracts key facets -- purposes, mechanisms, and evaluations -- from these and related papers, allowing users to interactively recombine facets to synthesize ideas. Scideator is driven by three design choices: (1) human-in-the-loop facet recombination, in which users select facets from retrieved papers and the system generates ideas by finding analogies across them via the Faceted Idea Generator module; (2) distance-controlled retrieval via the Analogous Paper Facet Finder module, which surfaces papers ranging from the same topic to entirely different areas to provide a spectrum of directions; and (3) facet-based novelty verification via the Idea Novelty Checker module, a retrieve-then-rerank pipeline that helps users to evaluate idea originality using facets. In a user study with computer science researchers, Scideator provided significantly more creativity support than a baseline using the same backbone LLM without our facet-based modules, particularly in idea exploration and expressiveness. Ablations further show that the facets benefit the novelty checker: facet-based retrieve-then-rerank surfaces more relevant papers than standard retrieval and re-ranking, and a facet-grounded novelty classifier outperforms classifiers that reason over unstructured ideas and papers.