mlsanityData to 14 Oct 2026

CRAWLDoc: A Dataset for Robust Ranking of Bibliographic Documents

arXiv 2506.03822 ↗First posted 4 Jun 2025cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Publication databases rely on accurate metadata extraction from diverse web sources, yet variations in web layouts and data formats present challenges for metadata providers. This paper introduces CRAWLDoc, a new method for contextual ranking of linked web documents. Starting with a publication's URL, such as a digital object identifier, CRAWLDoc retrieves the landing page and all linked web resources, including PDFs, ORCID profiles, and supplementary materials. It embeds these resources, along with anchor texts and the URLs, into a unified representation. For evaluating CRAWLDoc, we have created a new, manually labeled dataset of 600 publications from six top publishers in computer science. Our method CRAWLDoc demonstrates a robust and layout-independent ranking of relevant documents across publishers and data formats. It lays the foundation for improved metadata extraction from web documents with various layouts and formats. Our source code and dataset can be accessed at https://github.com/FKarl/CRAWLDoc.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

16 of 24 references.

References · 16

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  4. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  5. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  6. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  7. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  8. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  9. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 104
  10. 2019A Deep Look into Neural Ranking Models for Information Retrievalcited by 79
  11. 2019CEDR: Contextualized Embeddings for Document Rankingcited by 77
  12. 2023LMDX: Language Model-based Document Information Extraction and Localizationcited by 37
4 more
  1. 2020Text-based Question Answering from Information Retrieval and Deep Neural Network Perspectives: A Surveycited by 21
  2. 2024Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challengescited by 20
  3. 2016Neural Information Retrieval: A Literature Reviewcited by 4
  4. 2023Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documentscited by 4

Built on by

Not cited yet.