mlsanityData to 7 Oct 2026

MetaLead: A Comprehensive Human-Curated Leaderboard Dataset for Transparent Reporting of Machine Learning Experiments

arXiv 2601.22420 ↗First posted 30 Jan 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Leaderboards are crucial in the machine learning (ML) domain for benchmarking and tracking progress. However, creating leaderboards traditionally demands significant manual effort. In recent years, efforts have been made to automate leaderboard generation, but existing datasets for this purpose are limited by capturing only the best results from each paper and limited metadata. We present MetaLead, a fully human-annotated ML Leaderboard dataset that captures all experimental results for result transparency and contains extra metadata, such as the result experimental type: baseline, proposed method, or variation of proposed method for experiment-type guided comparisons, and explicitly separates train and test dataset for cross-domain assessment. This enriched structure makes MetaLead a powerful resource for more transparent and nuanced evaluations across ML research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

24 of 47 references.

References · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  3. 2018Datasheets for Datasetscited by 1,371
  4. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  5. 2018Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Constructioncited by 321
  6. 2018Construction of the Literature Graph in Semantic Scholarcited by 118
  7. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 116
  8. 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 111
  9. 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
  10. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  11. 2019Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Constructioncited by 50
  12. 2020AxCell: Automatic Extraction of Results from Machine Learning Paperscited by 48
12 more
  1. 2021TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metricscited by 29
  2. 2021End-to-End NLP Knowledge Graph Constructioncited by 21
  3. 2021Automated Mining of Leaderboards for Empirical AI Researchcited by 16
  4. 2023GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasetscited by 12
  5. 2023ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graphcited by 11
  6. 2024Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboardscited by 10
  7. 2018Automated Early Leaderboard Generation From Comparative Tablescited by 9
  8. 2018A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classificationcited by 7
  9. 2024Effective Context Selection in LLM-based Leaderboard Generation: An Empirical Studycited by 6
  10. 2024Position: Embracing Negative Results in Machine Learningcited by 5
  11. 2025A Position Paper on the Automatic Generation of Machine Learning Leaderboardscited by 3
  12. 2023NASA Science Mission Directorate Knowledge Graph Discoverycited by 2

Built on by

Not cited yet.