# ReaderLM-v2: Small Language Model for HTML to Markdown and JSON

arXiv: https://arxiv.org/abs/2503.01151 · HTML: https://mlsanity.com/paper/2503.01151/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Structured information extraction from complex scientific text with fine-tuned large language models](https://mlsanity.com/paper/2212.05238/) (2022)
- [HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems](https://mlsanity.com/paper/2411.02959/) (2024)
- [WebFormer: The Web-page Transformer for Structure Information Extraction](https://mlsanity.com/paper/2202.00217/) (2022)

## Built on by (most cited first, 15 of 15)

- [YourBench: Easy Custom Evaluation Sets for Everyone](https://mlsanity.com/paper/2504.01833/) (2025)
- [Safety Alignment via Constrained Knowledge Unlearning](https://mlsanity.com/paper/2505.18588/) (2025)
- [Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM](https://mlsanity.com/paper/2511.23119/) (2025)
- [SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glasses](https://mlsanity.com/paper/2602.22683/) (2026)
- [Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decoupling](https://mlsanity.com/paper/2505.17110/) (2025)
- [Reusing Pre-Training Data at Test Time is a Compute Multiplier](https://mlsanity.com/paper/2511.04234/) (2025)
- [Data Science and Technology Towards AGI Part I: Tiered Data Management](https://mlsanity.com/paper/2602.09003/) (2026)
- [An Index-based Approach for Efficient and Effective Web Content Extraction](https://mlsanity.com/paper/2512.06641/) (2025)
- [Region4Web: Rethinking Observation Space Granularity for Web Agents](https://mlsanity.com/paper/2605.07134/) (2026)
- [PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation](https://mlsanity.com/paper/2606.28344/) (2026)
- [Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents](https://mlsanity.com/paper/2609.22486/) (2026)
- [RedSage: A Cybersecurity Generalist LLM](https://mlsanity.com/paper/2601.22159/) (2026)
- [SCRIBES: Web-Scale Script-Based Semi-Structured Data Extraction with Reinforcement Learning](https://mlsanity.com/paper/2510.01832/) (2025)
- [news-crawler-LM: A Small Long-Context Model For High-Quality News Crawling](https://mlsanity.com/paper/2607.21284/) (2026)
- [Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards](https://mlsanity.com/paper/2609.03181/) (2026)

## Abstract (arXiv)

We present ReaderLM-v2, a compact 1.5 billion parameter language model designed for efficient web content extraction. Our model processes documents up to 512K tokens, transforming messy HTML into clean Markdown or JSON formats with high accuracy -- making it an ideal tool for grounding large language models. The model's effectiveness results from two key innovations: (1) a three-stage data synthesis pipeline that generates high quality, diverse training data by iteratively drafting, refining, and critiquing web content extraction; and (2) a unified training framework combining continuous pre-training with multi-objective optimization. Intensive evaluation demonstrates that ReaderLM-v2 outperforms GPT-4o-2024-08-06 and other larger models by 15-20% on carefully curated benchmarks, particularly excelling at documents exceeding 100K tokens, while maintaining significantly lower computational requirements.