mlsanityData to 14 Oct 2026

Submodular Benchmark Selection

arXiv 2605.02209 ↗First posted 4 May 2026cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Evaluating large language models across many benchmarks is expensive, yet many benchmarks are highly correlated. We formalize the selection of a small, informative subset as submodular maximization under a multivariate Gaussian model. Entropy (log-determinant covariance) and mutual information between selected and remaining benchmarks arise as natural objectives. Both are submodular; entropy selection coincides with pivoted Cholesky and has spectral residual bounds, while mutual information is non-monotone in general but empirically monotone for small subsets, so we optimize it greedily. Experiments on three matrices from ten public leaderboards show that mutual information selection outperforms entropy for imputation at small subsets.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

32 of 50 references.

References · 32

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
20 more
  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  8. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  10. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  11. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  12. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
  13. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  14. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
  15. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 201
  16. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 179
  17. 2014Lazier Than Lazy Greedycited by 99
  18. 2024Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBenchcited by 27
  19. 2024Sloth: scaling laws for LLM skills to predict multi-benchmark performance across familiescited by 24
  20. 2025One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Datacited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflowscited by 5
  2. 2026EarlyEval: Cheaper Agent Evaluation via Early Outcome Predictioncited by 3
  3. 2026Complement Submodular Information Measures for Balanced and Robust Data Selectioncited by 0
  4. 2026Balance of Benchmarks: Semantic Density Reweighting for Task-Conditioned Model Comparisoncited by 0
  5. 2026MatrixFormer: A Foundation Model for Matrix Completioncited by 0