# Boilerplate Removal using a Neural Sequence Labeling Model

arXiv: https://arxiv.org/abs/2004.14294 · HTML: https://mlsanity.com/paper/2004.14294/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2020.

## Built on by (most cited first, 11 of 11)

- [Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM](https://mlsanity.com/paper/2511.23119/) (2025)
- [Fundus: A Simple-to-Use News Scraper Optimized for High Quality Extractions](https://mlsanity.com/paper/2403.15279/) (2024)
- [ProVe: A Pipeline for Automated Provenance Verification of Knowledge Graphs against Textual Sources](https://mlsanity.com/paper/2210.14846/) (2022)
- [From Static to Dynamic: A Continual Learning Framework for Large Language Models](https://mlsanity.com/paper/2310.14248/) (2023)
- [PLAtE: A Large-scale Dataset for List Page Web Extraction](https://mlsanity.com/paper/2205.12386/) (2022)
- [An Index-based Approach for Efficient and Effective Web Content Extraction](https://mlsanity.com/paper/2512.06641/) (2025)
- [Boilerplate Detection via Semantic Classification of TextBlocks](https://mlsanity.com/paper/2203.04467/) (2022)
- [GROWN+UP: A Graph Representation Of a Webpage Network Utilizing Pre-training](https://mlsanity.com/paper/2208.02252/) (2022)
- [WCXB: A Multi-Type Web Content Extraction Benchmark](https://mlsanity.com/paper/2605.21097/) (2026)
- [news-crawler-LM: A Small Long-Context Model For High-Quality News Crawling](https://mlsanity.com/paper/2607.21284/) (2026)
- [PACE: Publisher-Adaptive Content Extraction via Agentic Automation](https://mlsanity.com/paper/2608.27466/) (2026)

## Abstract (arXiv)

The extraction of main content from web pages is an important task for numerous applications, ranging from usability aspects, like reader views for news articles in web browsers, to information retrieval or natural language processing. Existing approaches are lacking as they rely on large amounts of hand-crafted features for classification. This results in models that are tailored to a specific distribution of web pages, e.g. from a certain time frame, but lack in generalization power. We propose a neural sequence labeling model that does not rely on any hand-crafted features but takes only the HTML tags and words that appear in a web page as input. This allows us to present a browser extension which highlights the content of arbitrary web pages directly within the browser using our model. In addition, we create a new, more current dataset to show that our model is able to adapt to changes in the structure of web pages and outperform the state-of-the-art model.