mlsanityData to 7 Oct 2026

ReaderLM-v2: Small Language Model for HTML to Markdown and JSON

arXiv 2503.01151 ↗First posted 3 Mar 2025cs.CL cs.AI cs.IRcited by 15

Not on the map. Why?

Abstract (arXiv)

We present ReaderLM-v2, a compact 1.5 billion parameter language model designed for efficient web content extraction. Our model processes documents up to 512K tokens, transforming messy HTML into clean Markdown or JSON formats with high accuracy -- making it an ideal tool for grounding large language models. The model's effectiveness results from two key innovations: (1) a three-stage data synthesis pipeline that generates high quality, diverse training data by iteratively drafting, refining, and critiquing web content extraction; and (2) a unified training framework combining continuous pre-training with multi-objective optimization. Intensive evaluation demonstrates that ReaderLM-v2 outperforms GPT-4o-2024-08-06 and other larger models by 15-20% on carefully curated benchmarks, particularly excelling at documents exceeding 100K tokens, while maintaining significantly lower computational requirements.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

15 of 24 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  5. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  6. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  7. 2020Efficient Transformers: A Surveycited by 784
  8. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  10. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  11. 2022A Contrastive Framework for Neural Text Generationcited by 224
  12. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
3 more
  1. 2022Structured information extraction from complex scientific text with fine-tuned large language modelscited by 56
  2. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  3. 2022WebFormer: The Web-page Transformer for Structure Information Extractioncited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025YourBench: Easy Custom Evaluation Sets for Everyonecited by 20
  2. 2025Safety Alignment via Constrained Knowledge Unlearningcited by 13
  3. 2025Dripper: Token-Efficient Main HTML Extraction with a Lightweight LMcited by 8
  4. 2026SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glassescited by 6
  5. 2025Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decouplingcited by 5
  6. 2025Reusing Pre-Training Data at Test Time is a Compute Multipliercited by 4
  7. 2025An Index-based Approach for Efficient and Effective Web Content Extractioncited by 2
  8. 2026Data Science and Technology Towards AGI Part I: Tiered Data Managementcited by 2
  9. 2026PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generationcited by 2
  10. 2026Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agentscited by 2
  11. 2026RedSage: A Cybersecurity Generalist LLMcited by 1
  12. 2026Region4Web: Rethinking Observation Space Granularity for Web Agentscited by 1
3 more
  1. 2025SCRIBES: Web-Scale Script-Based Semi-Structured Data Extraction with Reinforcement Learningcited by 0
  2. 2026news-crawler-LM: A Small Long-Context Model For High-Quality News Crawlingcited by 0
  3. 2026Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining
  2. 2024Rethinking KenLM: Good and Bad Model Ensembles for Efficient Text Quality Filtering in Large Web Corpora
  3. 2024Leveraging Large Language Models for Web Scraping
  4. 2026Webscraper: Leverage Multimodal Large Language Models for Index-Content Web Scraping
  5. 2026WCXB: A Multi-Type Web Content Extraction Benchmark
  6. 2025Benchmarking MLLM-based Web Understanding: Reasoning, Robustness and Safety
  7. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
  8. 2025Blu-WERP (Web Extraction and Refinement Pipeline): A Scalable Pipeline for Preprocessing Large Language Model Datasets