# CoVA: Context-aware Visual Attention for Webpage Information Extraction

arXiv: https://arxiv.org/abs/2110.12320 · HTML: https://mlsanity.com/paper/2110.12320/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 61 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning](https://mlsanity.com/paper/1909.02151/) (2019)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [One-Shot Object Detection with Co-Attention and Co-Excitation](https://mlsanity.com/paper/1911.12529/) (2019)
- [Spatial Dependency Parsing for Semi-Structured Document Information Extraction](https://mlsanity.com/paper/2005.00642/) (2020)
- [Jointly Learning Explainable Rules for Recommendation with Knowledge Graph](https://mlsanity.com/paper/1903.03714/) (2019)
- [FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents](https://mlsanity.com/paper/2010.10755/) (2020)
- [Simplified DOM Trees for Transferable Attribute Extraction from the Web](https://mlsanity.com/paper/2101.02415/) (2021)
- [Corpus Conversion Service: A Machine Learning Platform to Ingest Documents at Scale](https://mlsanity.com/paper/1806.02284/) (2018)
- [Attention-based Joint Detection of Object and Semantic Part](https://mlsanity.com/paper/2007.02419/) (2020)
- [Deep Reader: Information extraction from Document images via relation extraction and Natural Language](https://mlsanity.com/paper/1812.04377/) (2018)

## Built on by (most cited first, 6 of 6)

- [WinClick: GUI Grounding with Multimodal Large Language Models](https://mlsanity.com/paper/2503.04730/) (2025)
- [Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis](https://mlsanity.com/paper/2511.12658/) (2025)
- [The Klarna Product Page Dataset: Web Element Nomination with Graph Neural Networks and Large Language Models](https://mlsanity.com/paper/2111.02168/) (2021)
- [Deep-TEMPEST: Using Deep Learning to Eavesdrop on HDMI from its Unintended Electromagnetic Emanations](https://mlsanity.com/paper/2407.09717/) (2024)
- [Multilingual Attribute Extraction from News Web Pages](https://mlsanity.com/paper/2502.02167/) (2025)
- [Visual-Aware Representation of Web Pages for Machine Learning Applications](https://mlsanity.com/paper/2608.18727/) (2026)

## Abstract (arXiv)

Webpage information extraction (WIE) is an important step to create knowledge bases. For this, classical WIE methods leverage the Document Object Model (DOM) tree of a website. However, use of the DOM tree poses significant challenges as context and appearance are encoded in an abstract manner. To address this challenge we propose to reformulate WIE as a context-aware Webpage Object Detection task. Specifically, we develop a Context-aware Visual Attention-based (CoVA) detection pipeline which combines appearance features with syntactical structure from the DOM tree. To study the approach we collect a new large-scale dataset of e-commerce websites for which we manually annotate every web element with four labels: product price, product title, product image and background. On this dataset we show that the proposed CoVA approach is a new challenging baseline which improves upon prior state-of-the-art methods.