Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolution
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts. Iterative multi-turn prompting often leads to context degradation and diminishing cognitive returns. We present PAWNI (Prompt Architecture Wizard using Neural Intelligence), an agentic conversational interface of eight agents that transforms unstructured queries into structured prompts through guided question-and-answer dialogue informed by a self-evolving knowledge base. Rather than optimising the model's response, PAWNI optimises the question itself by front-loading intent clarification. We also propose a three-tier framework of 18 prompt elements across Essential, Enhancement, and Elevation categories. To evaluate system behaviour and validate a measurement protocol, we conducted an exploratory within-subjects study (N=4) across four complex tasks, integrating 32-channel EEG, NASA-TLX workload, and behavioural metrics. Participants produced more structurally complete prompts with PAWNI (42% to 91% of assessed elements), rated LLM outputs higher across all quality dimensions, and reported lower workload (39.6 vs. 21.7 NASA-TLX). Every participant reached satisfactory output in a single turn, compared to 1-12 turns unaided. While effect sizes are unstable due to sample size, direction consistency supports the hypothesis that optimising prompt formulation front-end is a critical lever for human-AI collaboration.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 69 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
35 of 69 references.
Closest ancestors · 3
Same direction, newest first
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan
- 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application23 Dec
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
Other references · 32
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023A Survey of Large Language Modelscited by 2,537
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
- 2023Large Language Models as Optimizerscited by 830
20 more
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
- 2024TextGrad: Automatic "Differentiation" via Textcited by 250
- 2023Universal Self-Consistency for Large Language Model Generationcited by 198
- 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 115
- 2025Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Taskcited by 103
- 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 99
- 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
- 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
- 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 24
- 2025Effects of Prompt Length on Domain-specific Tasks for Large Language Modelscited by 10
- 2024A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Educationcited by 8
- 2025Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Findcited by 7
- 2025Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoningcited by 5
- 2023Thinking Assistants: LLM-Based Conversational Assistants that Help Users Think By Asking rather than Answeringcited by 4
- 2026Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirerscited by 3
- 2025ChatGPT produces more "lazy" thinkers: Evidence of cognitive engagement declinecited by 2
- 2026Autonomous Question Formation for Large Language Model-Driven AI Systemscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization
- 2026Helix: A Dual-Helix Co-Evolutionary Multi-Agent System for Prompt Optimization and Question Reformulation
- 2025First Ask Then Answer: A Framework Design for AI Dialogue Based on Supplementary Questioning with Large Language Models
- 2025Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models
- 2025DynamicMind: A Tri-Mode Thinking System for Large Language Models
- 2024Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning
- 2025Benchmarking Contextual Understanding for In-Car Conversational Systems
- 2026Memory as an Attack Surface in LLM Agents: A Study on Multiple-Choice Question Answering