mlsanityData to 7 Oct 2026

CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent

arXiv 2310.06155 ↗First posted 9 Oct 2023cs.HC cs.CEcited by 34

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Developing novel research questions (RQs) often requires extensive literature reviews, especially in interdisciplinary fields. To support RQ development through human-AI co-creation, we leveraged Large Language Models (LLMs) to build an LLM-based agent system named CoQuest. We conducted an experiment with 20 HCI researchers to examine the impact of two interaction designs: breadth-first and depth-first RQ generation. The findings revealed that participants perceived the breadth-first approach as more creative and trustworthy upon task completion. Conversely, during the task, participants considered the depth-first generated RQs as more creative. Additionally, we discovered that AI processing delays allowed users to reflect on multiple RQs simultaneously, leading to a higher quantity of generated RQs and an enhanced sense of control. Our work makes both theoretical and practical contributions by proposing and evaluating a mental model for human-AI co-creation of RQs. We also address potential ethical issues, such as biases and over-reliance on AI, advocating for using the system to improve human research creativity rather than automating scientific inquiry.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 102 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

38 of 102 references.

References · 38

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  12. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
26 more
  1. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  2. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  3. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  4. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  5. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  6. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  7. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
  8. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  9. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  10. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  11. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  12. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  13. 2023Aligning Language Models to User Opinionscited by 98
  14. 2023Art and the science of generative AI: A deeper divecited by 93
  15. 2022Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systemscited by 59
  16. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  17. 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
  18. 2022How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Modelscited by 43
  19. 2023Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Casescited by 32
  20. 2019Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creatorscited by 31
  21. 2023Toward General Design Principles for Generative AI Applicationscited by 20
  22. 2023Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systemscited by 20
  23. 2022When happy accidents spark creativity: Bringing collaborative speculation to life with generative AIcited by 9
  24. 2022Augmenting Scientific Creativity with Retrieval across Knowledge Domainscited by 7
  25. 2023The Right Variety: Improving Expressive Range Analysis with Metric Selection Methodscited by 6
  26. 2020The societal and ethical relevance of computational creativitycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 15; 2026: 5.

34 papers.

Sort:

By area:

  1. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
  2. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  3. 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 40
  4. 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
  5. 2024Assessing and Understanding Creativity in Large Language Modelscited by 35
  6. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  7. 2024'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participantscited by 27
  8. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  9. 2024CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraintscited by 22
  10. 2024PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideationcited by 16
  11. 2025IRIS: Interactive Research Ideation System for Accelerating Scientific Discoverycited by 16
  12. 2023Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agentscited by 13
22 more
  1. 2025SPHERE: An Evaluation Card for Human-AI Systemscited by 9
  2. 2025CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experimentscited by 8
  3. 2024Creativity in AI: Progresses and Challengescited by 7
  4. 2024Language Models as Critical Thinking Tools: A Case Study of Philosopherscited by 6
  5. 2025XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaborationcited by 6
  6. 2025Interaction as Intelligence: Deep Research With Human-AI Partnershipcited by 5
  7. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  8. 2025ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inferencecited by 4
  9. 2024Randomness Is All You Need: Semantic Traversal of Problem-Solution Spaces with Large Language Modelscited by 3
  10. 2025QG-SMS: Enhancing Test Item Analysis via Student Modeling and Simulationcited by 3
  11. 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
  12. 2025XAgen: An Explainability Tool for Identifying and Correcting Failures in Multi-Agent Workflowscited by 3
  13. 2025The Design Space of Recent AI-assisted Research Tools for Ideation, Sensemaking, and Scientific Creativitycited by 2
  14. 2026Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideationcited by 2
  15. 2025AnimAgents: Coordinating Multi-Stage Animation Pre-Production with Human-Multi-Agent Collaborationcited by 1
  16. 2026Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Taskscited by 1
  17. 2025Resurrecting Socrates in the Age of AI: A Study Protocol for Evaluating a Socratic Tutor to Support Research Question Development in Higher Educationcited by 0
  18. 2025PRAC3 (Privacy, Reputation, Accountability, Consent, Credit, Compensation): Long Tailed Risks of Voice Actors in AI Data-Economycited by 0
  19. 2025LAPPI: Interactive Optimization with LLM-Assisted Preference-Based Problem Instantiationcited by 0
  20. 2026Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Controlcited by 0
  21. 2026ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentationcited by 0
  22. 2026Position: AI Agents in Scientific Teams Should Be Studied as Human-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Orchestrating LLM Agents for Scientific Research: A Pilot Study of Multiple Choice Question (MCQ) Generation and Evaluation
  2. 2025Exploring Design of Multi-Agent LLM Dialogues for Research Ideation
  3. 2026AgentPanel: Toward a New Paradigm for Human--AI Collaboration in Exploring Scientific Questions
  4. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents
  5. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
  6. 2025"Shall We Dig Deeper?": Designing and Evaluating Strategies for LLM Agents to Advance Knowledge Co-Construction in Asynchronous Online Discussions
  7. 2023Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents
  8. 2026FirstResearch: Auditable Question Formation for LLM Scientific Discovery Agents