{"list":"other","total":71,"rows":[{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"2004.07780","title":"Shortcut Learning in Deep Neural Networks","created":"2020-04-16","cited_by_count":1969},{"id":"2104.08663","title":"BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models","created":"2021-04-17","cited_by_count":1252},{"id":"2307.03109","title":"A Survey on Evaluation of Large Language Models","created":"2023-07-06","cited_by_count":1242},{"id":"2310.03693","title":"Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!","created":"2023-10-05","cited_by_count":1154},{"id":"1810.03993","title":"Model Cards for Model Reporting","created":"2018-10-05","cited_by_count":1080},{"id":"2311.17035","title":"Scalable Extraction of Training Data from (Production) Language Models","created":"2023-11-28","cited_by_count":434},{"id":"2412.14093","title":"Alignment faking in large language models","created":"2024-12-18","cited_by_count":360},{"id":"2111.15366","title":"AI and the Everything in the Whole Wide World Benchmark","created":"2021-11-26","cited_by_count":349},{"id":"2401.00595","title":"State of What Art? A Call for Multi-Prompt LLM Evaluation","created":"2023-12-31","cited_by_count":310},{"id":"2412.04984","title":"Frontier Models are Capable of In-context Scheming","created":"2024-12-06","cited_by_count":287},{"id":"2012.05345","title":"Data and its (dis)contents: A survey of dataset development and use in machine learning research","created":"2020-12-09","cited_by_count":260},{"id":"2203.08242","title":"Data Contamination: From Memorization to Exploitation","created":"2022-03-15","cited_by_count":236},{"id":"2205.10770","title":"Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models","created":"2022-05-22","cited_by_count":220},{"id":"2407.01502","title":"AI Agents That Matter","created":"2024-07-01","cited_by_count":219},{"id":"1909.12475","title":"Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging","created":"2019-09-27","cited_by_count":216},{"id":"2104.02145","title":"What Will it Take to Fix Benchmarking in Natural Language Understanding?","created":"2021-04-05","cited_by_count":203},{"id":"2311.04850","title":"Rethinking Benchmark and Contamination for Language Models with Rephrased Samples","created":"2023-11-08","cited_by_count":195},{"id":"2406.04127","title":"Are We Done with MMLU?","created":"2024-06-06","cited_by_count":194},{"id":"2405.14782","title":"Lessons from the Trenches on Reproducible Evaluation of Language Models","created":"2024-05-23","cited_by_count":179},{"id":"2310.11986","title":"Sociotechnical Safety Evaluation of Generative AI Systems","created":"2023-10-18","cited_by_count":157},{"id":"2202.06935","title":"Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text","created":"2022-02-14","cited_by_count":155},{"id":"2406.04244","title":"Benchmark Data Contamination of Large Language Models: A Survey","created":"2024-06-06","cited_by_count":147},{"id":"2406.07358","title":"AI Sandbagging: Language Models can Strategically Underperform on Evaluations","created":"2024-06-11","cited_by_count":145},{"id":"2008.02637","title":"Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets","created":"2020-08-06","cited_by_count":138},{"id":"2402.01781","title":"When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards","created":"2024-02-01","cited_by_count":137},{"id":"2107.07002","title":"The Benchmark Lottery","created":"2021-07-14","cited_by_count":136},{"id":"2401.14446","title":"Black-Box Access is Insufficient for Rigorous AI Audits","created":"2024-01-25","cited_by_count":126},{"id":"1910.01500","title":"MLPerf Training Benchmark","created":"2019-10-02","cited_by_count":115},{"id":"2411.12990","title":"BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices","created":"2024-11-20","cited_by_count":114},{"id":"2406.03689","title":"Evaluating the World Model Implicit in a Generative Model","created":"2024-06-06","cited_by_count":113},{"id":"2404.18824","title":"Benchmarking Benchmark Leakage in Large Language Models","created":"2024-04-29","cited_by_count":106},{"id":"2112.01716","title":"Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research","created":"2021-12-03","cited_by_count":102},{"id":"2306.04618","title":"Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations","created":"2023-06-07","cited_by_count":97},{"id":"2005.11295","title":"From ImageNet to Image Classification: Contextualizing Progress on Benchmarks","created":"2020-05-22","cited_by_count":90},{"id":"2108.04308","title":"Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development","created":"2021-08-09","cited_by_count":85},{"id":"2404.05399","title":"SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety","created":"2024-04-08","cited_by_count":77},{"id":"2203.04592","title":"Mapping global dynamics of benchmark creation and saturation in artificial intelligence","created":"2022-03-09","cited_by_count":71},{"id":"2402.09880","title":"Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence","created":"2024-02-15","cited_by_count":68},{"id":"2106.05498","title":"It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks","created":"2021-06-10","cited_by_count":61},{"id":"2407.21792","title":"Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?","created":"2024-07-31","cited_by_count":61},{"id":"2306.03100","title":"Rethinking Model Evaluation as Narrowing the Socio-Technical Gap","created":"2023-06-01","cited_by_count":56},{"id":"2104.06387","title":"ExplainaBoard: An Explainable Leaderboard for NLP","created":"2021-04-13","cited_by_count":50},{"id":"2205.05256","title":"Evaluation Gaps in Machine Learning Practice","created":"2022-05-11","cited_by_count":49},{"id":"2007.04792","title":"Targeting the Benchmark: On Methodology in Current Natural Language Processing Research","created":"2020-07-07","cited_by_count":45},{"id":"2007.07399","title":"Bringing the People Back In: Contesting Benchmark Machine Learning Datasets","created":"2020-07-14","cited_by_count":43},{"id":"2401.14462","title":"AI auditing: The Broken Bus on the Road to AI Accountability","created":"2024-01-25","cited_by_count":43},{"id":"2402.12649","title":"Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation","created":"2024-02-20","cited_by_count":41},{"id":"2310.10628","title":"Data Contamination Through the Lens of Time","created":"2023-10-16","cited_by_count":40},{"id":"2007.03488","title":"Benchmarking in Optimization: Best Practice and Open Issues","created":"2020-07-07","cited_by_count":38},{"id":"2402.17861","title":"Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling","created":"2024-02-27","cited_by_count":29},{"id":"2208.12852","title":"What Do NLP Researchers Believe? Results of the NLP Community Metasurvey","created":"2022-08-26","cited_by_count":28},{"id":"2305.09022","title":"It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance","created":"2023-05-15","cited_by_count":26},{"id":"2410.07959","title":"COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act","created":"2024-10-10","cited_by_count":24},{"id":"2205.08363","title":"REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research","created":"2022-05-05","cited_by_count":20},{"id":"2501.07458","title":"Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI","created":"2025-01-13","cited_by_count":19},{"id":"2407.05694","title":"On the Limitations of Compute Thresholds as a Governance Strategy","created":"2024-07-08","cited_by_count":18},{"id":"2201.07040","title":"Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals","created":"2022-01-18","cited_by_count":16},{"id":"2410.08385","title":"Language model developers should report train-test overlap","created":"2024-10-10","cited_by_count":16},{"id":"2204.05151","title":"Metaethical Perspectives on 'Benchmarking' AI Ethics","created":"2022-04-11","cited_by_count":12},{"id":"2404.06647","title":"From Protoscience to Epistemic Monoculture: How Benchmarking Set the Stage for the Deep Learning Revolution","created":"2024-04-09","cited_by_count":12},{"id":"2407.09221","title":"Evaluating AI Evaluation: Perils and Prospects","created":"2024-07-12","cited_by_count":11},{"id":"2412.15184","title":"Data for Mathematical Copilots: Better Ways of Presenting Proofs for Machine Learning","created":"2024-12-19","cited_by_count":11},{"id":"2209.00692","title":"Making Intelligence: Ethical Values in IQ and ML Benchmarks","created":"2022-09-01","cited_by_count":10},{"id":"2404.17293","title":"Lazy Data Practices Harm Fairness Research","created":"2024-04-26","cited_by_count":10},{"id":"2410.11672","title":"Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers","created":"2024-10-15","cited_by_count":9},{"id":"2412.08392","title":"The Roles of English in Evaluating Multilingual Language Models","created":"2024-12-11","cited_by_count":7},{"id":"2409.00252","title":"Building Better Datasets: Seven Recommendations for Responsible Design from Dataset Creators","created":"2024-08-30","cited_by_count":5},{"id":"2407.12220","title":"Questionable practices in machine learning","created":"2024-07-17","cited_by_count":4},{"id":"2411.05224","title":"Beyond the Numbers: Transparency in Relation Extraction Benchmark Creation and Leaderboards","created":"2024-11-07","cited_by_count":2}],"next_cursor":null}