mlsanityData to 14 Oct 2026

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

arXiv 2609.10297 ↗First posted 9 Sep 2026cs.CVcited by 0

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

GUI agents accumulate high-resolution screenshots as the trajectory unfolds, increasing inference latency and memory usage. Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint. Once tokens are discarded, the corresponding visual evidence cannot be recovered without re-encoding. Pruning therefore becomes an textitirreversible admission decision that must remain useful for unknown future targets while preserving coverage of operable regions under tight budgets. To address these challenges, we propose textbfmethod, a training-free framework for emphTrajectory-robust Admission and Coverage-aware Evidence ordering. Specifically, we combine a query-independent layout-derived interaction prior with instruction relevance and feature novelty to rank visual evidence according to both potential future utility and diversity. Then, we reserve part of the budget for native visual tokens distributed across the screen, repairing missing spatial coverage without breaking the ordering. Together, these mechanisms produce a nested token order, allowing retained visual evidence to shrink monotonically across budgets while remaining reusable throughout the trajectory. Finally, our monotone KV contraction incrementally contracts retired frames into compact session state, avoiding repeated visual encoding or pruning. Extensive experiments across six GUI benchmarks and diverse models verify the effectiveness of our proposed method under tight budgets. The source code will be released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 39 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

35 of 39 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026AQuaUI: Visual Token Reduction for GUI Agents with Adaptive Quadtrees19 May
  2. 2026ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction11 May
  3. 2026OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments3 Apr
  4. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents15 Feb
  5. 2025GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness1 Oct
  6. 2025PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents1 Oct
  7. 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents25 Jul
  8. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  10. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  11. 2024On the Effects of Data Scale on UI Control Agents6 Jun
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
2 more
  1. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents17 Jan
  2. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 21

  1. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  4. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  5. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  6. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  7. 2024OmniParser for Pure Vision Based GUI Agentcited by 198
  8. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Modelscited by 181
  9. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 157
  10. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMscited by 96
  11. 2024Matryoshka Multimodal Modelscited by 84
  12. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Modelcited by 57
9 more
  1. 2024MMInA: Benchmarking Multihop Multimodal Internet Agentscited by 55
  2. 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Accelerationcited by 41
  3. 2025SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMscited by 24
  4. 2026Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversitycited by 24
  5. 2026VisionTrim: Unified Vision Token Compression for Training-Free MLLM Accelerationcited by 18
  6. 2025SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inferencecited by 15
  7. 2026IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruningcited by 12
  8. 2026Spatio-Temporal Token Pruning for Efficient High-Resolution GUI Agentscited by 7
  9. 2026STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Where and How to Prune: An Empirical Study of Visual Token Pruning for GUI Agent Navigation
  2. 2026STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI Agents
  3. 2026CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI Agents
  4. 2026ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agents
  5. 2026Recurrent Latent Visual Search for GUI Grounding
  6. 2026Compress to Focus: Efficient Coordinate Compression for Policy Optimization in Multi-Turn GUI Agents
  7. 2026QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning
  8. 2025Visual Test-time Scaling for GUI Agent Grounding