# SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis

arXiv: https://arxiv.org/abs/2601.18305 · HTML: https://mlsanity.com/paper/2601.18305/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 30 of 30)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [UI-TARS: Pioneering Automated GUI Interaction with Native Agents](https://mlsanity.com/paper/2501.12326/) (2025)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents](https://mlsanity.com/paper/2405.14573/) (2024)
- [AppAgent: Multimodal Agents as Smartphone Users](https://mlsanity.com/paper/2312.13771/) (2023)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use](https://mlsanity.com/paper/2504.07981/) (2025)
- [GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents](https://mlsanity.com/paper/2504.10458/) (2025)
- [ShowUI: One Vision-Language-Action Model for GUI Visual Agent](https://mlsanity.com/paper/2411.17465/) (2024)
- [OmniParser for Pure Vision Based GUI Agent](https://mlsanity.com/paper/2408.00203/) (2024)
- [GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices](https://mlsanity.com/paper/2406.08451/) (2024)
- [Mapping Natural Language Instructions to Mobile UI Action Sequences](https://mlsanity.com/paper/2005.03776/) (2020)
- [UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning](https://mlsanity.com/paper/2503.21620/) (2025)
- [GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation](https://mlsanity.com/paper/2311.07562/) (2023)
- [AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents](https://mlsanity.com/paper/2407.17490/) (2024)
- [MAI-UI Technical Report: Real-World Centric Foundation GUI Agents](https://mlsanity.com/paper/2512.22047/) (2025)
- [GUI-G2: Gaussian Reward Modeling for GUI Grounding](https://mlsanity.com/paper/2507.15846/) (2025)
- [A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility](https://mlsanity.com/paper/2202.02312/) (2022)
- [AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.01391/) (2025)
- [GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent](https://mlsanity.com/paper/2505.16827/) (2025)
- [InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization](https://mlsanity.com/paper/2508.05731/) (2025)
- [UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoning](https://mlsanity.com/paper/2510.20286/) (2025)
- [Morae: Proactively Pausing UI Agents for User Choices](https://mlsanity.com/paper/2508.21456/) (2025)
- [GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning](https://mlsanity.com/paper/2401.14268/) (2024)
- [AUITestAgent: Automatic Requirements Oriented GUI Function Testing](https://mlsanity.com/paper/2407.09018/) (2024)
- [Agent for User: Testing Multi-User Interactive Features in TikTok](https://mlsanity.com/paper/2504.15474/) (2025)
- [BTL-UI: Blink-Think-Link Reasoning Model for GUI Agent](https://mlsanity.com/paper/2509.15566/) (2025)

## Abstract (arXiv)

Despite numerous Graphical User Interface (GUI) agents claiming to automate user interaction tasks, to date, few achieve satisfactory interaction capability with human users in real-world scenarios. Through empirical analysis, this paper identifies the root cause of the limited interaction capability as the rigid swipe execution. In particular, unlike humans, who perform swipes with fine-grained control over trajectory, speed, and timing, existing agents can only conduct simplistic, deterministic swipe behaviors, leading to frequent failures on complicated user-like interaction tasks. Due to the lack of open-source human-like swipe training data, we propose SwipeGen, the first tool for synthesizing diverse and human-like swipe interactions, and SwipeBench, the first benchmark for evaluating agents' swipe interaction quality. Extensive experiments show that SwipeGen can improve the swipe execution success rate of existing agents by up to 2.46x. Our code, dataset, and model are available at https://github.com/TSKGHS17/SwipeGen.