mlsanityData to 7 Oct 2026

VisCritic: Visual State Comparison as Process Reward for GUI Agents

arXiv 2606.24525 ↗First posted 23 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 2 more
Abstract (arXiv)

GUI agents powered by vision-language models show strong potential for automating digital tasks, yet frequently fail in long-horizon scenarios due to the absence of step-level verification. Existing process reward models verify actions through textual reasoning alone, missing the visual nature of GUI state changes. We introduce VisCritic, a visual process reward framework that verifies agent actions by directly comparing pre-action and post-action screenshots in visual feature space. VisCritic employs a Siamese vision transformer to extract change-aware representations, coupled with an Action-Aware Critic Head that jointly evaluates action success, task progress, and error type. A critic-training data construction pipeline generates weakly supervised samples from existing trajectories without additional human labels for critic training. Experiments and offline analyses across five benchmarks demonstrate that VisCritic serves as a plug-and-play enhancement for diverse GUI agents, generally improving benchmark metrics while providing visual diagnostic cues.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 50 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

44 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation5 Jun
  2. 2025Guiding VLM Agents with Process Rewards at Inference Time for GUI Navigation22 Apr
  3. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  4. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec

Other references · 40

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2025Qwen2.5-VL Technical Reportcited by 5,418
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  10. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  12. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
28 more
  1. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  2. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  3. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  4. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  5. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  6. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  7. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  8. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  9. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  10. 2018Fully Convolutional Siamese Networks for Change Detectioncited by 305
  11. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  12. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  13. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  14. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
  15. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  16. 2025BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanismcited by 25
  17. 2025GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Taskscited by 25
  18. 2026MobileDreamer: Generative Sketch World Model for GUI Agentcited by 21
  19. 2024Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves?cited by 20
  20. 2026Computer-Using World Modelcited by 16
  21. 2026Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agentscited by 12
  22. 2026Adaptive Milestone Reward for GUI Agentscited by 11
  23. 2025GUI-PRA: Process Reward Agent for GUI Taskscited by 8
  24. 2025Dyna-Mind: Learning to Simulate from Experience for Better AI Agentscited by 7
  25. 2025CRAFT-GUI: Curriculum-Reinforced Agent For GUI Taskscited by 6
  26. 2026Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoningcited by 4
  27. 2026Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbationscited by 3
  28. 2026Do Multi-Agents Dream of Electric Screens? Achieving Perfect Accuracy on AndroidWorld Through Task Decompositioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GUI-Eyes: Tool-Augmented Perception for Visual Grounding in GUI Agents
  2. 2026Scaling GUI Agents with Visual State Transitions
  3. 2026GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models
  4. 2026SkillLens: Visual Skill Cards for Retrieval-Augmented GUI Action Prediction and On-Policy Distillation
  5. 2026MagicGUI-RMS: A Multi-Agent Reward Model System for Self-Evolving GUI Agents via Automated Feedback Reflux
  6. 2026EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding
  7. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agents
  8. 2026Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training