# CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent

arXiv: https://arxiv.org/abs/2310.06155 · HTML: https://mlsanity.com/paper/2310.06155/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 38 of its 102 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 38)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [The case for 4-bit precision: k-bit Inference Scaling Laws](https://mlsanity.com/paper/2212.09720/) (2022)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Sources of Hallucination by Large Language Models on Inference Tasks](https://mlsanity.com/paper/2305.14552/) (2023)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems](https://mlsanity.com/paper/2204.07666/) (2022)
- [Metacognitive Prompting Improves Understanding in Large Language Models](https://mlsanity.com/paper/2308.05342/) (2023)
- [Investigating Explainability of Generative AI for Code through Scenario-based Design](https://mlsanity.com/paper/2202.04903/) (2022)
- [How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models](https://mlsanity.com/paper/2209.01390/) (2022)

## Built on by (most cited first, 30 of 34)

- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback](https://mlsanity.com/paper/2410.04025/) (2024)
- [Cocoa: Co-Planning and Co-Execution with AI Agents](https://mlsanity.com/paper/2412.10999/) (2024)
- [Assessing and Understanding Creativity in Large Language Models](https://mlsanity.com/paper/2401.12491/) (2024)
- [Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents](https://mlsanity.com/paper/2405.10467/) (2024)
- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery](https://mlsanity.com/paper/2504.16728/) (2025)
- [PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation](https://mlsanity.com/paper/2409.12538/) (2024)
- [Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents](https://mlsanity.com/paper/2311.13148/) (2023)
- [SPHERE: An Evaluation Card for Human-AI Systems](https://mlsanity.com/paper/2504.07971/) (2025)
- [CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experiments](https://mlsanity.com/paper/2509.13588/) (2025)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaboration](https://mlsanity.com/paper/2505.11336/) (2025)
- [Language Models as Critical Thinking Tools: A Case Study of Philosophers](https://mlsanity.com/paper/2404.04516/) (2024)
- [Interaction as Intelligence: Deep Research With Human-AI Partnership](https://mlsanity.com/paper/2507.15759/) (2025)
- [A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model](https://mlsanity.com/paper/2408.02920/) (2024)
- [ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference](https://mlsanity.com/paper/2509.14537/) (2025)
- [Randomness Is All You Need: Semantic Traversal of Problem-Solution Spaces with Large Language Models](https://mlsanity.com/paper/2402.06053/) (2024)
- [The Design Space of Recent AI-assisted Research Tools for Ideation, Sensemaking, and Scientific Creativity](https://mlsanity.com/paper/2502.16291/) (2025)
- [QG-SMS: Enhancing Test Item Analysis via Student Modeling and Simulation](https://mlsanity.com/paper/2503.05888/) (2025)
- [Reimagining Urban Science: Scaling Causal Inference with Large Language Models](https://mlsanity.com/paper/2504.12345/) (2025)
- [XAgen: An Explainability Tool for Identifying and Correcting Failures in Multi-Agent Workflows](https://mlsanity.com/paper/2512.17896/) (2025)
- [Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation](https://mlsanity.com/paper/2604.28031/) (2026)
- [PRAC3 (Privacy, Reputation, Accountability, Consent, Credit, Compensation): Long Tailed Risks of Voice Actors in AI Data-Economy](https://mlsanity.com/paper/2507.16247/) (2025)
- [AnimAgents: Coordinating Multi-Stage Animation Pre-Production with Human-Multi-Agent Collaboration](https://mlsanity.com/paper/2511.17906/) (2025)
- [Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Tasks](https://mlsanity.com/paper/2603.19504/) (2026)
- [Resurrecting Socrates in the Age of AI: A Study Protocol for Evaluating a Socratic Tutor to Support Research Question Development in Higher Education](https://mlsanity.com/paper/2504.06294/) (2025)

## Abstract (arXiv)

Developing novel research questions (RQs) often requires extensive literature reviews, especially in interdisciplinary fields. To support RQ development through human-AI co-creation, we leveraged Large Language Models (LLMs) to build an LLM-based agent system named CoQuest. We conducted an experiment with 20 HCI researchers to examine the impact of two interaction designs: breadth-first and depth-first RQ generation. The findings revealed that participants perceived the breadth-first approach as more creative and trustworthy upon task completion. Conversely, during the task, participants considered the depth-first generated RQs as more creative. Additionally, we discovered that AI processing delays allowed users to reflect on multiple RQs simultaneously, leading to a higher quantity of generated RQs and an enhanced sense of control. Our work makes both theoretical and practical contributions by proposing and evaluating a mental model for human-AI co-creation of RQs. We also address potential ethical issues, such as biases and over-reliance on AI, advocating for using the system to improve human research creativity rather than automating scientific inquiry.