mlsanityData to 7 Oct 2026

League: Leaderboard Generation on Demand

arXiv 2502.18209 ↗First posted 25 Feb 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence (AI). Faced with a large number of AI papers updated daily, it becomes difficult for researchers to track every paper's proposed methods, experimental results, and settings, prompting the need for efficient automatic leaderboard construction. While large language models (LLMs) offer promise in automating this process, challenges such as multi-document summarization, leaderboard generation, and experiment fair comparison still remain under exploration. LAG solves these challenges through a systematic approach that involves the paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation. Our contributions include a comprehensive solution to the leaderboard construction problem, a reliable evaluation method, and experimental results showing the high quality of leaderboards.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 23 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

19 of 23 references.

References · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  4. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  5. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  6. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
  7. 2023Augmenting Language Models with Long-Term Memorycited by 186
  8. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 174
  9. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  10. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 116
  11. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  12. 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
7 more
  1. 2019Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Constructioncited by 50
  2. 2020AxCell: Automatic Extraction of Results from Machine Learning Paperscited by 48
  3. 2023ORKG-Leaderboards: A Systematic Workflow for Mining Leaderboards as a Knowledge Graphcited by 11
  4. 2024Efficient Performance Tracking: Leveraging Large Language Models for Automated Construction of Scientific Leaderboardscited by 10
  5. 2024SDCL: Students Discrepancy-Informed Correction Learning for Semi-supervised Medical Image Segmentationcited by 9
  6. 2023All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extractioncited by 2
  7. 2024LEGOBench: Scientific Leaderboard Generation Benchmarkcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025How Far Are AI Scientists from Changing the World?cited by 15