# DOM-Q-NET: Grounded RL on Structured Language

arXiv: https://arxiv.org/abs/1902.07257 · HTML: https://mlsanity.com/paper/1902.07257/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2019.

## Built on by (most cited first, 25 of 25)

- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023)
- [Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security](https://mlsanity.com/paper/2401.05459/) (2024)
- [Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control](https://mlsanity.com/paper/2306.07863/) (2023)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [A data-driven approach for learning to control computers](https://mlsanity.com/paper/2202.08137/) (2022)
- [From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces](https://mlsanity.com/paper/2306.00245/) (2023)
- [AgentStudio: A Toolkit for Building General Virtual Agents](https://mlsanity.com/paper/2403.17918/) (2024)
- [OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use](https://mlsanity.com/paper/2508.04482/) (2025)
- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://mlsanity.com/paper/2311.18751/) (2023)
- [Dual-View Visual Contextualization for Web Navigation](https://mlsanity.com/paper/2402.04476/) (2024)
- [WebVLN: Vision-and-Language Navigation on Websites](https://mlsanity.com/paper/2312.15820/) (2023)
- [A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding](https://mlsanity.com/paper/2305.03668/) (2023)
- [AppBuddy: Learning to Accomplish Tasks in Mobile Apps via Reinforcement Learning](https://mlsanity.com/paper/2106.00133/) (2021)
- [WebWISE: Web Interface Control and Sequential Exploration with Large Language Models](https://mlsanity.com/paper/2310.16042/) (2023)
- [Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTs](https://mlsanity.com/paper/2203.07828/) (2022)
- [Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization](https://mlsanity.com/paper/2205.12648/) (2022)
- [The Klarna Product Page Dataset: Web Element Nomination with Graph Neural Networks and Large Language Models](https://mlsanity.com/paper/2111.02168/) (2021)
- [Skim: Speculative Execution for Fast and Efficient Web Agents](https://mlsanity.com/paper/2605.16565/) (2026)
- [Learning Context-Aware Representations of Subtrees](https://mlsanity.com/paper/2111.04308/) (2021)
- [MacroBench: A Novel Testbed for Web Automation Scripts via Large Language Models](https://mlsanity.com/paper/2510.04363/) (2025)
- [TVWorld: Foundations for Remote-Control TV Agents](https://mlsanity.com/paper/2601.13142/) (2026)
- [HxAgent: Iterative Agent Planning for End-to-End Web Application Testing](https://mlsanity.com/paper/2608.15491/) (2026)

## Abstract (arXiv)

Building agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficult for current deep reinforcement learning (RL) models due to the large discrete action space and the varying number of actions between the states. In this work, we introduce DOM-Q-NET, a novel architecture for RL-based web navigation to address both of these problems. It parametrizes Q functions with separate networks for different action categories: clicking a DOM element and typing a string input. Our model utilizes a graph neural network to represent the tree-structured HTML of a standard web page. We demonstrate the capabilities of our model on the MiniWoB environment where we can match or outperform existing work without the use of expert demonstrations. Furthermore, we show 2x improvements in sample efficiency when training in the multi-task setting, allowing our model to transfer learned behaviours across tasks.