# On the Use of ArXiv as a Dataset

arXiv: https://arxiv.org/abs/1905.00075 · HTML: https://mlsanity.com/paper/1905.00075/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2019.

## Built on by (most cited first, 30 of 103)

- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset](https://mlsanity.com/paper/2303.03915/) (2023)
- [Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models](https://mlsanity.com/paper/2403.00231/) (2024)
- [EuroLLM: Multilingual Language Models for Europe](https://mlsanity.com/paper/2409.16235/) (2024)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models](https://mlsanity.com/paper/2503.23350/) (2025)
- [SciCap: Generating Captions for Scientific Figures](https://mlsanity.com/paper/2110.11624/) (2021)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [User Inference Attacks on Large Language Models](https://mlsanity.com/paper/2310.09266/) (2023)
- [OCR-VQGAN: Taming Text-within-Image Generation](https://mlsanity.com/paper/2210.11248/) (2022)
- [Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?](https://mlsanity.com/paper/2309.16595/) (2023)
- [AnthroScore: A Computational Linguistic Measure of Anthropomorphism](https://mlsanity.com/paper/2402.02056/) (2024)
- [RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling](https://mlsanity.com/paper/2105.06597/) (2021)
- [SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation](https://mlsanity.com/paper/2110.10774/) (2021)
- [Mid-Training of Large Language Models: A Survey](https://mlsanity.com/paper/2510.06826/) (2025)
- [Predicting vs. Acting: A Trade-off Between World Modeling & Agent Modeling](https://mlsanity.com/paper/2407.02446/) (2024)
- [Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design](https://mlsanity.com/paper/2410.19123/) (2024)
- [Towards Better Evaluation of Instruction-Following: A Case-Study in Summarization](https://mlsanity.com/paper/2310.08394/) (2023)
- [MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation](https://mlsanity.com/paper/2407.00468/) (2024)
- [Large Language Models Encode Semantics and Alignment in Linearly Separable Representations](https://mlsanity.com/paper/2507.09709/) (2025)
- [BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models](https://mlsanity.com/paper/2402.11573/) (2024)
- [Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMs](https://mlsanity.com/paper/2407.04411/) (2024)
- [Text summarization via global structure awareness](https://mlsanity.com/paper/2602.09821/) (2026)
- [Online Label Shift: Optimal Dynamic Regret meets Practical Algorithms](https://mlsanity.com/paper/2305.19570/) (2023)
- [Source Attribution for Large Language Model-Generated Data](https://mlsanity.com/paper/2310.00646/) (2023)
- [KV Prediction for Improved Time to First Token](https://mlsanity.com/paper/2410.08391/) (2024)
- [Adaptive Neural Message Passing for Inductive Learning on Hypergraphs](https://mlsanity.com/paper/2109.10683/) (2021)
- [SIGMA: An Efficient Heterophilous Graph Neural Network with Fast Global Aggregation](https://mlsanity.com/paper/2305.09958/) (2023)
- [Studying word order through iterative shuffling](https://mlsanity.com/paper/2109.04867/) (2021)

## Abstract (arXiv)

The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print features text, figures, authors, citations, categories, and other metadata. These rich, multi-modal features, combined with the natural graph structure---created by citation, affiliation, and co-authorship---makes the arXiv an exciting candidate for benchmarking next-generation models. Here we take the first necessary steps toward this goal, by providing a pipeline which standardizes and simplifies access to the arXiv's publicly available data. We use this pipeline to extract and analyze a 6.7 million edge citation graph, with an 11 billion word corpus of full-text research articles. We present some baseline classification results, and motivate application of more exciting generative graph models.