mlsanityData to 14 Oct 2026

Cocoa: Co-Planning and Co-Execution with AI Agents

arXiv 2412.10999 ↗First posted 14 Dec 2024cs.HC cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

As AI agents take on increasingly long-running tasks involving sophisticated planning and execution, there is a corresponding need for novel interaction designs that enable deeper human-agent collaboration. However, most prior works leverage human interaction to fix "autonomous" workflows that have yet to become fully autonomous or rigidly treat planning and execution as separate stages. Based on a formative study with 9 researchers using AI to support their work, we propose a design that affords greater flexibility in collaboration, so that users can 1) delegate agency to the user or agent via a collaborative plan where individual steps can be assigned; and 2) interleave planning and execution so that plans can adjust after partial execution. We introduce Cocoa, a system that takes design inspiration from computational notebooks to support complex research tasks. A lab study (n=16) found that Cocoa enabled steerability without sacrificing ease-of-use, and a week-long field deployment (n=7) showed how researchers collaborated with Cocoa to accomplish real-world tasks.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 58 of its 160 references that are in the map. First posted on arXiv 14 Dec 2024.

Builds on

58 of 160 references.

References · 58

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  7. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  8. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  9. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  10. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  11. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  12. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
38 more
  1. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 312
  2. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  3. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  4. 2024AI Agents That Mattercited by 219
  5. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 135
  6. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 105
  7. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 100
  8. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  9. 2024Visibility into AI Agentscited by 79
  10. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  11. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  12. 2024Building Machines that Learn and Think with Peoplecited by 73
  13. 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
  14. 2024LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptionscited by 47
  15. 2023VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototypingcited by 46
  16. 2024IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedbackcited by 42
  17. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  18. 2024Data-driven Discovery with Large Generative Modelscited by 34
  19. 2023Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Promptscited by 30
  20. 2024Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decompositioncited by 29
  21. 2022Scim: Intelligent Skimming Support for Scientific Paperscited by 27
  22. 2023LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMscited by 27
  23. 2023Synergi: A Mixed-Initiative System for Scholarly Synthesis and Sensemakingcited by 26
  24. 2022A Computational Inflection for Scientific Discoverycited by 23
  25. 2022Threddy: An Interactive System for Personalized Thread-based Exploration and Organization of Scientific Literaturecited by 22
  26. 2023CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Contextcited by 22
  27. 2023Interactive AI Alignment: Specification, Process, and Evaluation Alignmentcited by 22
  28. 2024NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptionscited by 22
  29. 2023Relatedly: Scaffolding Literature Reviews with Existing Related Work Sectionscited by 21
  30. 2024MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflowscited by 19
  31. 2023Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responsescited by 18
  32. 2024PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Paperscited by 18
  33. 2023Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflowscited by 17
  34. 2024Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoningcited by 16
  35. 2022Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Workcited by 15
  36. 2023How does HCI Understand Human Autonomy and Agency?cited by 14
  37. 2023VAL: Interactive Task Learning with GPT Dialog Parsingcited by 14
  38. 2020mage: Fluid Moves Between Code and Graphical Work in Computational Notebookscited by 13

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 25.

39 papers.

Sort:

By area:

  1. 2025Levels of Autonomy for AI Agentscited by 48
  2. 2025Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversightcited by 16
  3. 2025Thinking beyond the anthropomorphic paradigm benefits LLM researchcited by 14
  4. 2025Measuring and mitigating overreliance to build human-compatible AIcited by 12
  5. 2026Mapping the Design Space of User Experience for Computer Use Agentscited by 7
  6. 2025Measuring LLM Novelty As The Frontier Of Original And High-Quality Outputcited by 6
  7. 2025A Good Plan is Hard to Find: Aligning Models with Preferences is Misaligned with What Helps Userscited by 6
  8. 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Taskscited by 6
  9. 2026Comparing Human Oversight Strategies for Computer-Use Agentscited by 6
  10. 2025TableTalk: Scaffolding Spreadsheet Development with a Language Agentcited by 5
  11. 2026Modeling Distinct Human Interaction in Web Agentscited by 5
  12. 2025ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inferencecited by 4
27 more
  1. 2025Semantic Commit: Helping Users Update Intent Specifications for AI Memory at Scalecited by 3
  2. 2025Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answerscited by 3
  3. 2025On the Regulatory Potential of User Interfaces for AI Agent Governancecited by 3
  4. 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunitiescited by 3
  5. 2026Hedwig: Dynamic Autonomy for Coding Agents Under Local Oversightcited by 3
  6. 2026Plover: Steering GUI Agents through Plan-Centric Interactioncited by 3
  7. 2025Understanding User Experiences of Computer Use Agents: Design Space and Opportunities for Building Agent UX Prototypescited by 2
  8. 2026How to Steer Your Multi-Agent System: Human-LLM Collaborative Planningcited by 2
  9. 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualizationcited by 2
  10. 2026"When to Hand Off, When to Work Together": Understanding Concurrent Human-Agent Interaction in Shared Co-Creative Workspacescited by 1
  11. 2026LitPivot: Developing Well-Situated Research Ideas Through Dynamic Contextualization and Critique within the Literature Landscapecited by 1
  12. 2026Effects of Generative AI Errors on User Reliance Across Task Difficultycited by 1
  13. 2026Auditing and Controlling AI Agent Actions in Spreadsheetscited by 1
  14. 2026Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discoverycited by 1
  15. 2026CollabSkill: Evaluating Human-Agent Collaboration On Real-World Taskscited by 1
  16. 2026HANSEL: Extracting Breadcrumbs from Web Agent Trajectories for Interactive Verificationcited by 1
  17. 2026Agent-Integrated Software: Interaction Contracts and Continuous Assurancecited by 1
  18. 2025OrchVis: Hierarchical Multi-Agent Orchestration for Human Oversightcited by 0
  19. 2026How Researchers Navigate Accountability, Transparency, and Trust When Using AI Tools in Early-Stage Research: A Think-Aloud Studycited by 0
  20. 2026"What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Usecited by 0
  21. 2026Creativity from Friction: Human-AI Interaction for Exploratory Structural Designcited by 0
  22. 2026Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agentscited by 0
  23. 2026AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysiscited by 0
  24. 2026Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scalecited by 0
  25. 2026Trust and Task Completion in the World of Consumer AI Agentscited by 0
  26. 2026Ream: Unfolding Mutual Awareness in Human-Agent Workspacescited by 0
  27. 2026Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Executioncited by 0