mlsanityData to 14 Oct 2026

LocalLSTC: A Long Short-Term Control Architecture for Locally Deployed GUI Agents

arXiv 2608.25777 ↗First posted 26 Aug 2026cs.AIcited by 0

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Modern GUI-agent frameworks achieve strong desktop task performance with frontier API models, yet persistent control information often remains implicit in growing interaction trajectories. At each step, the planner reconstructs the active task stage, accumulated evidence, and runtime feedback before deciding the next action. This dependence becomes more pronounced under weaker local reasoning backbones. Across four representative state-of-the-art frameworks, replacing GPT-5 with Qwen3.5-9B reduces average OSWorld SR-100 from 60.9% to 35.2%. Trajectory annotation further identifies at least one control failure in 94.7% of failed trajectories. To address this problem, we introduce LocalLSTC, a training-free architecture that organizes control by temporal scope, maintaining persistent cross-step state to guide short-term execution commitments. The framework comprises two complementary mechanisms. Long-to-Short Planning forms each commitment from persistent state, while Short-to-Long Control integrates execution outcomes back into that state for progress assessment, recovery, and termination. With Qwen3.8-27B, LocalLSTC reaches 73.7% SR-100 on OSWorld and 68.5% on WindowsAgentArena, achieving the strongest local OSWorld result and a new state of the art on WindowsAgentArena. Ablations further support contributions from both mechanisms. Together, these results show that temporal organization of control information can reduce cross-step control failures and complement backbone scaling in locally deployed GUI agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 47 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

35 of 47 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Plover: Steering GUI Agents through Plan-Centric Interaction16 Jul
  2. 2026VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation23 Apr
  3. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents13 Apr
  4. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
  5. 2025AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Management11 Dec
  6. 2025UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability21 Nov
  7. 2025Scaling Agents for Computer Use2 Oct
  8. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  9. 2025ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents19 Aug
  10. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  11. 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning2 Jun
  12. 2025Agent-SAMA: State-Aware Mobile Assistant29 May
6 more
  1. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents1 Apr
  2. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  3. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  4. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
  5. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr

Other references · 17

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  4. 2025Kimi-VL Technical Reportcited by 388
  5. 2025Seed1.5-VL Technical Reportcited by 373
  6. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  7. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  8. 2025GTA1: GUI Test-time Scaling Agentcited by 106
  9. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 103
  10. 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
  11. 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 35
  12. 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agentscited by 27
5 more
  1. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 21
  2. 2025Mano Technical Reportcited by 13
  3. 2026A Subgoal-driven Framework for Improving Long-Horizon LLM Agentscited by 11
  4. 2025Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestrationcited by 8
  5. 2026Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Expertscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learn from Move: the Next Step for GUI Agents
  2. 2026A Task-State Representation for Long-Horizon Mobile GUI Agents
  3. 2025Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI Automation
  4. 2026GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  5. 2026GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL
  6. 2026GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
  7. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
  8. 2025Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent