mlsanityData to 14 Oct 2026

StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure

arXiv 2607.11388 ↗First posted 13 Jul 2026cs.AI cs.CL cs.LGcited by 7

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled increasingly capable digital agents for computer use. However, real-world tasks are often long-horizon and involve evolving contexts containing accumulated observations, intermediate edits, failed attempts, and partially completed executions. Existing agents typically operate over raw interaction history, making task progress difficult to interpret, verify, and recover, which ultimately limits reliable long-horizon execution. In this paper, we argue that addressing this challenge requires explicitly structuring both the agent's state and workflow around a unified causal representation of task progress. We present StructAgent, a state-centered framework that introduces a unified state for maintaining compact, verifiable task progress and a structured workflow that regulates progress through verifier-backed state transitions. Building on this design, StructAgent further enables explicit progress checkpointing, evidence-driven task completion, targeted failure recovery, and tool-supported execution, while ensuring that all progress updates remain grounded in verification. Extensive experiments demonstrate that StructAgent consistently improves a wide range of LLM and VLM backbones on long-horizon computer-use tasks. On OSWorld-Verified, it improves Qwen3.5-9B from 27.0% to 46.9% success rate and Qwen3.5-27B from 31.6% to 62.2%, while achieving a new open-source state of the art of 78.9% with MiniMax-M3. Moreover, the same framework generalizes beyond desktop environments to Minecraft, demonstrating the generality of our design.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 54 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

51 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Hybrid Self-evolving Structured Memory for GUI Agents11 Mar
  2. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent12 Jan
  3. 2025EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration22 Dec
  4. 2025MGA: Memory-Driven GUI Agent for Observation-Centric Interaction28 Oct
  5. 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data18 Sep
  6. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  7. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience6 Aug
  8. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct

Other references · 43

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2023A Survey of Large Language Modelscited by 2,576
  4. 2017Billion-scale similarity search with GPUscited by 2,497
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  8. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  9. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  10. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  11. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  12. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
31 more
  1. 2024Agent Workflow Memorycited by 383
  2. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  3. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  4. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  5. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  6. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  7. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
  8. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 171
  9. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 159
  10. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 117
  11. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108
  12. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 97
  13. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 84
  14. 2025Contextual Experience Replay for Self-Improvement of Language Agentscited by 48
  15. 2026MiniMax Sparse Attentioncited by 44
  16. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  17. 2025Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deploymentcited by 38
  18. 2025GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Taskscited by 26
  19. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 23
  20. 2025CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agentcited by 18
  21. 2025Scaling Agents for Computer Usecited by 14
  22. 2026Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correctioncited by 14
  23. 2026The Art of Building Verifiers for Computer Use Agentscited by 13
  24. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentscited by 11
  25. 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agentscited by 10
  26. 2024MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memorycited by 9
  27. 2025GUI-PRA: Process Reward Agent for GUI Taskscited by 9
  28. 2026MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge Evolutioncited by 9
  29. 2026Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolutioncited by 7
  30. 2025VerificAgent: Domain-Specific Memory Verification for Scalable Oversight of Aligned Computer-Use Agentscited by 3
  31. 2025"Are We Done Yet?": A Vision-Based Judge for Autonomous Task Completion of Computer Use Agentscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Interlock: From Local Correctness to Task Completion in Long-Horizon Agents
  2. 2026Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents
  3. 2026LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
  4. 2026InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents
  5. 2026The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents
  6. 2026Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
  7. 2026Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability
  8. 2026Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States