mlsanityData to 14 Oct 2026

TVWorld: Foundations for Remote-Control TV Agents

arXiv 2601.13142 ↗First posted 19 Jan 2026cs.CV cs.AI cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Recent large vision-language models (LVLMs) have demonstrated strong potential for device control. However, existing research has primarily focused on point-and-click (PnC) interaction, while remote-control (RC) interaction commonly encountered in everyday TV usage remains largely underexplored. To fill this gap, we introduce TVWorld, an offline graph-based abstraction of real-world TV navigation that enables reproducible and deployment-free evaluation. On this basis, we derive two complementary benchmarks that comprehensively assess TV-use capabilities: TVWorld-N for topology-aware navigation and TVWorld-G for focus-aware grounding. These benchmarks expose a key limitation of existing agents: insufficient topology awareness for focus-based, long-horizon TV navigation. Motivated by this finding, we propose a Topology-Aware Training framework that injects topology awareness into LVLMs. Using this framework, we develop TVTheseus, a foundation model specialized for TV navigation. TVTheseus achieves a success rate of 68.3% on TVWorld-N, surpassing strong closed-source baselines such as Gemini 3 Flash and establishing state-of-the-art (SOTA) performance. Additional analyses further provide valuable insights into the development of effective TV-use agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 74 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

45 of 74 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control27 Aug
  2. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  3. 2025ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search21 May
  4. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
  5. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  6. 2025Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study21 Mar
  7. 2025AppVLM: A Lightweight Vision Language Model for Online App Control10 Feb
  8. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  9. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct
  10. 2024CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents1 Jul
  11. 2024GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding16 Jun
  12. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understanding7 Feb

Other references · 33

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  4. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  5. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  6. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  7. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  8. 2017DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoningcited by 266
  9. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  10. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  11. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  12. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 187
21 more
  1. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
  2. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 168
  3. 2017Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learningcited by 161
  4. 2024Android in the Zoo: Chain-of-Action-Thought for GUI Agentscited by 159
  5. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
  6. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 81
  7. 2025Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignmentcited by 79
  8. 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replaycited by 77
  9. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
  10. 2018Learning to Navigate the Webcited by 60
  11. 2024Ferret-UI 2: Mastering Universal User Interface Understanding Across Platformscited by 51
  12. 2025UFO2: The Desktop AgentOScited by 41
  13. 2025GUI-Shepherd: Reliable Process Reward and Verification for Long-Sequence GUI Taskscited by 26
  14. 2019DOM-Q-NET: Grounded RL on Structured Languagecited by 25
  15. 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21
  16. 2024Ponder & Press: Advancing Visual GUI Agent towards General Computer Controlcited by 20
  17. 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
  18. 2025COLA: A Scalable Multi-Agent Framework For Windows UI Task Automationcited by 9
  19. 2025STEVE: A Step Verification Pipeline for Computer-use Agent Trainingcited by 5
  20. 2025Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agentscited by 4
  21. 2023Training Automated Defense Strategies Using Graph-based Cyber Attack Simulationscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigation
  2. 2026TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation
  3. 2026Robostral Navigate
  4. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
  5. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  6. 2025ETP-R1: Evolving Topological Planning with Reinforcement Fine-tuning for Vision-Language Navigation in Continuous Environments
  7. 2026SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
  8. 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning