TVWorld: Foundations for Remote-Control TV Agents
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Recent large vision-language models (LVLMs) have demonstrated strong potential for device control. However, existing research has primarily focused on point-and-click (PnC) interaction, while remote-control (RC) interaction commonly encountered in everyday TV usage remains largely underexplored. To fill this gap, we introduce TVWorld, an offline graph-based abstraction of real-world TV navigation that enables reproducible and deployment-free evaluation. On this basis, we derive two complementary benchmarks that comprehensively assess TV-use capabilities: TVWorld-N for topology-aware navigation and TVWorld-G for focus-aware grounding. These benchmarks expose a key limitation of existing agents: insufficient topology awareness for focus-based, long-horizon TV navigation. Motivated by this finding, we propose a Topology-Aware Training framework that injects topology awareness into LVLMs. Using this framework, we develop TVTheseus, a foundation model specialized for TV navigation. TVTheseus achieves a success rate of 68.3% on TVWorld-N, surpassing strong closed-source baselines such as Gemini 3 Flash and establishing state-of-the-art (SOTA) performance. Additional analyses further provide valuable insights into the development of effective TV-use agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 74 references that are in the map. First posted on arXiv 19 Jan 2026.
Builds on
45 of 74 references.
Closest ancestors · 12
Same direction, newest first
- 2025SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control27 Aug
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
- 2025ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search21 May
- 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
- 2025Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study21 Mar
- 2025AppVLM: A Lightweight Vision Language Model for Online App Control10 Feb
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
- 2024CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents1 Jul
- 2024GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding16 Jun
- 2024ScreenAI: A Vision-Language Model for UI and Infographics Understanding7 Feb
Other references · 33
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
- 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
- 2017DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoningcited by 266
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
- 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 187
21 more
- 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 168
- 2017Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learningcited by 161
- 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
- 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
- 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 81
- 2025Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignmentcited by 79
- 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replaycited by 77
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
- 2018Learning to Navigate the Webcited by 60
- 2024Ferret-UI 2: Mastering Universal User Interface Understanding Across Platformscited by 51
- 2025UFO2: The Desktop AgentOScited by 41
- 2025GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Taskscited by 26
- 2019DOM-Q-NET: Grounded RL on Structured Languagecited by 25
- 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21
- 2024Ponder & Press: Advancing Visual GUI Agent towards General Computer Controlcited by 20
- 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
- 2025COLA: A Scalable Multi-Agent Framework For Windows UI Task Automationcited by 9
- 2025STEVE: A Step Verification Pipeline for Computer-use Agent Trainingcited by 5
- 2025Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agentscited by 4
- 2023Training Automated Defense Strategies Using Graph-based Cyber Attack Simulationscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
- 2026TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation
- 2026Robostral Navigate
- 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
- 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
- 2025ETP-R1: Evolving Topological Planning with Reinforcement Fine-tuning for Vision-Language Navigation in Continuous Environments
- 2026SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
- 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning