mlsanityData to 7 Oct 2026

Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces

arXiv 2309.14459 ↗First posted 25 Sep 2023cs.HCcited by 71

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) exhibit dynamic capabilities and appear to comprehend complex and ambiguous natural language prompts. However, calibrating LLM interactions is challenging for interface designers and end-users alike. A central issue is our limited grasp of how human cognitive processes begin with a goal and form intentions for executing actions, a blindspot even in established interaction models such as Norman's gulfs of execution and evaluation. To address this gap, we theorize how end-users 'envision' translating their goals into clear intentions and craft prompts to obtain the desired LLM response. We define a process of Envisioning by highlighting three misalignments: (1) knowing whether LLMs can accomplish the task, (2) how to instruct the LLM to do the task, and (3) how to evaluate the success of the LLM's output in meeting the goal. Finally, we make recommendations to narrow the envisioning gulf in human-LLM interactions.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Sep 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 33; 2026: 32.

71 papers.

Sort:

By area:

  1. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  2. 2024WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualizationcited by 38
  3. 2024Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualizationcited by 23
  4. 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 19
  5. 2025Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Reviewcited by 16
  6. 2024LLM Whisperer: An Inconspicuous Attack to Bias LLM Responsescited by 15
  7. 2025Thinking beyond the anthropomorphic paradigm benefits LLM researchcited by 13
  8. 2025HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistantscited by 10
  9. 2025Measuring and mitigating overreliance to build human-compatible AIcited by 9
  10. 2026Human-AI Co-design for Clinical Prediction Modelscited by 9
  11. 2025AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Toolscited by 8
  12. 2025CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experimentscited by 8
38 more
  1. 2024Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiencycited by 7
  2. 2025OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Modelscited by 7
  3. 2025Just-In-Time Objectives: A General Approach for Specialized AI Interactionscited by 7
  4. 2026AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wildcited by 7
  5. 2025DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Designcited by 6
  6. 2025Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation Workflowscited by 6
  7. 2025Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creationcited by 6
  8. 2025Do It For Me vs. Do It With Me: Investigating User Perceptions of Different Paradigms of Automation in Copilots for Feature-Rich Softwarecited by 5
  9. 2025Orca: Browsing at Scale Through User-Driven and AI-Facilitated Orchestration Across Malleable Webpagescited by 4
  10. 2026From Words to Widgets for Controllable LLM Generationcited by 4
  11. 2024CogErgLLM: Exploring Large Language Model Systems Design Perspective Using Cognitive Ergonomicscited by 3
  12. 2025From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Taskscited by 3
  13. 2025Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfacescited by 3
  14. 2025NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modificationcited by 3
  15. 2025Interaction-Driven Browsing: A Human-in-the-Loop Conceptual Framework Informed by Human Web Browsing for Browser-Using Agentscited by 3
  16. 2025ALLOY: Generating Reusable Agent Workflows from User Demonstrationcited by 3
  17. 2026DiscoverLLM: From Executing Intents to Discovering Themcited by 3
  18. 2026Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversationscited by 3
  19. 2025ACAI for SBOs: AI Co-creation for Advertising and Inspiration for Small Business Ownerscited by 2
  20. 2025Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMscited by 2
  21. 2025When the Domain Expert Has No Time and the LLM Developer Has No Clinical Expertise: Real-World Lessons from LLM Co-Design in a Safety-Net Hospitalcited by 2
  22. 2025Understanding User Experiences of Computer Use Agents: Design Space and Opportunities for Building Agent UX Prototypescited by 2
  23. 2026Texterial: A Text-as-Material Interaction Paradigm for LLM-Mediated Writingcited by 2
  24. 2026More Than "Means to an End": Supporting Reasoning with Transparently Designed AI Data Science Processescited by 2
  25. 2026AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Usecited by 2
  26. 2024Generative AI in Medicinecited by 1
  27. 2025Lessons for GenAI Literacy From a Field Study of Human-GenAI Augmentation in the Workplacecited by 1
  28. 2025AIAP: A No-Code Workflow Builder for Non-Experts with Natural Language and Multi-Agent Collaborationcited by 1
  29. 2025Psychologically Enhanced AI Agentscited by 1
  30. 2026Bridging Gulfs in UI Generation through Semantic Guidancecited by 1
  31. 2026Chaplains' Reflections on the Design and Usage of AI for Conversational Carecited by 1
  32. 2026Effects of Generative AI Errors on User Reliance Across Task Difficultycited by 1
  33. 2026Creo: From One-Shot Image Generation to Progressive, Co-Creative Ideationcited by 1
  34. 2026Auditing and Controlling AI Agent Actions in Spreadsheetscited by 1
  35. 2026Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discoverycited by 1
  36. 2026Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deploymentscited by 1
  37. 2025VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agentscited by 0
  38. 2025Human-AI collaboration or obedient and often clueless AI in instruct, serve, repeat dynamics?cited by 0

50 most cited of 71 All 71 (JSON)