{"total":310,"rows":[{"id":"2402.14992","title":"tinyBenchmarks: evaluating LLMs with fewer examples","created":"2024-02-22","cited_by_count":310,"area":null},{"id":"2401.03065","title":"CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution","created":"2024-01-05","cited_by_count":300,"area":null},{"id":"2405.14782","title":"Lessons from the Trenches on Reproducible Evaluation of Language Models","created":"2024-05-23","cited_by_count":179,"area":null},{"id":"2410.12405","title":"ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs","created":"2024-10-16","cited_by_count":149,"area":null},{"id":"2401.12187","title":"WARM: On the Benefits of Weight Averaged Reward Models","created":"2024-01-22","cited_by_count":147,"area":"alignment"},{"id":"2402.01781","title":"When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards","created":"2024-02-01","cited_by_count":137,"area":null},{"id":"2505.13995","title":"ELEPHANT: Measuring and understanding social sycophancy in LLMs","created":"2025-05-20","cited_by_count":129,"area":null},{"id":"2402.01536","title":"Homogenization Effects of Large Language Models on Human Creative Ideation","created":"2024-02-02","cited_by_count":127,"area":null},{"id":"2511.04703","title":"Measuring what Matters: Construct Validity in Large Language Model Benchmarks","created":"2025-11-03","cited_by_count":119,"area":null},{"id":"2407.10457","title":"The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism","created":"2024-07-15","cited_by_count":116,"area":null},{"id":"2408.02442","title":"Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models","created":"2024-08-05","cited_by_count":96,"area":null},{"id":"2401.06766","title":"Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements","created":"2024-01-12","cited_by_count":83,"area":null},{"id":"2405.17202","title":"Efficient multi-prompt evaluation of LLMs","created":"2024-05-27","cited_by_count":73,"area":null},{"id":"2502.06559","title":"Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation","created":"2025-02-10","cited_by_count":60,"area":null},{"id":"2410.02185","title":"POSIX: A Prompt Sensitivity Index For Large Language Models","created":"2024-10-03","cited_by_count":59,"area":null},{"id":"2402.17649","title":"Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs","created":"2024-02-27","cited_by_count":55,"area":null},{"id":"2410.13025","title":"LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks","created":"2024-10-16","cited_by_count":52,"area":null},{"id":"2410.16491","title":"BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data","created":"2024-10-21","cited_by_count":50,"area":null},{"id":"2407.03129","title":"Social Bias Evaluation for Large Language Models Requires Prompt Variations","created":"2024-07-03","cited_by_count":40,"area":null},{"id":"2406.10248","title":"On the Worst Prompt Performance of Large Language Models","created":"2024-06-08","cited_by_count":38,"area":null},{"id":"2510.27106","title":"Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworks","created":"2025-10-31","cited_by_count":38,"area":null},{"id":"2410.06458","title":"LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints","created":"2024-10-09","cited_by_count":35,"area":"lr"},{"id":"2509.11106","title":"Fluid Language Model Benchmarking","created":"2025-09-14","cited_by_count":35,"area":null},{"id":"2401.14493","title":"K-QA: A Real-World Medical Q&A Benchmark","created":"2024-01-25","cited_by_count":31,"area":null},{"id":"2404.16966","title":"Examining the robustness of LLM evaluation to the distributional assumptions of benchmarks","created":"2024-04-25","cited_by_count":31,"area":null},{"id":"2509.01790","title":"Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs","created":"2025-09-01","cited_by_count":30,"area":"lr"},{"id":"2406.15708","title":"Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization","created":"2024-06-22","cited_by_count":29,"area":null},{"id":"2411.14258","title":"Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective","created":"2024-11-21","cited_by_count":29,"area":null},{"id":"2412.09569","title":"JuStRank: Benchmarking LLM Judges for System Ranking","created":"2024-12-12","cited_by_count":29,"area":null},{"id":"2502.11266","title":"The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models","created":"2025-02-16","cited_by_count":29,"area":null},{"id":"2502.21228","title":"ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer","created":"2025-02-28","cited_by_count":29,"area":null},{"id":"2406.06196","title":"LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages","created":"2024-06-10","cited_by_count":28,"area":"lr"},{"id":"2407.13696","title":"Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench","created":"2024-07-18","cited_by_count":27,"area":null},{"id":"2502.12964","title":"Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer","created":"2025-02-18","cited_by_count":27,"area":null},{"id":"2404.18923","title":"Holmes: A Benchmark to Assess the Linguistic Competence of Language Models","created":"2024-04-29","cited_by_count":26,"area":null},{"id":"2504.10356","title":"MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages","created":"2025-04-14","cited_by_count":26,"area":null},{"id":"2409.18786","title":"A Survey on the Honesty of Large Language Models","created":"2024-09-27","cited_by_count":23,"area":"alignment"},{"id":"2410.18697","title":"How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs","created":"2024-10-24","cited_by_count":23,"area":null},{"id":"2408.10646","title":"Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs","created":"2024-08-20","cited_by_count":21,"area":null},{"id":"2503.01622","title":"DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation","created":"2025-03-03","cited_by_count":21,"area":null},{"id":"2504.06969","title":"Towards LLMs Robustness to Changes in Prompt Format Styles","created":"2025-04-09","cited_by_count":21,"area":null},{"id":"2603.13285","title":"Brittlebench: Quantifying LLM robustness via prompt sensitivity","created":"2026-02-27","cited_by_count":21,"area":null},{"id":"2402.19475","title":"The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?","created":"2024-02-29","cited_by_count":20,"area":null},{"id":"2509.03581","title":"Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents","created":"2025-09-03","cited_by_count":20,"area":"lr"},{"id":"2402.07891","title":"Label-Efficient Model Selection for Text Generation","created":"2024-02-12","cited_by_count":19,"area":null},{"id":"2502.11767","title":"From Selection to Generation: A Survey of LLM-based Active Learning","created":"2025-02-17","cited_by_count":19,"area":null},{"id":"2503.00137","title":"SCORE: Systematic COnsistency and Robustness Evaluation for Large Language Models","created":"2025-02-28","cited_by_count":19,"area":null},{"id":"2412.12004","title":"The Open Source Advantage in Large Language Models (LLMs)","created":"2024-12-16","cited_by_count":18,"area":null},{"id":"2505.12189","title":"Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steering","created":"2025-05-18","cited_by_count":18,"area":"lr"},{"id":"2404.05904","title":"The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models","created":"2024-04-08","cited_by_count":17,"area":null},{"id":"2504.02733","title":"Enhancing LLM Robustness to Perturbed Instructions: An Empirical Study","created":"2025-04-03","cited_by_count":17,"area":null},{"id":"2406.18528","title":"PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation","created":"2024-06-26","cited_by_count":16,"area":null},{"id":"2502.04295","title":"Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization","created":"2025-02-06","cited_by_count":16,"area":null},{"id":"2507.22168","title":"Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles","created":"2025-07-29","cited_by_count":16,"area":null},{"id":"2602.00521","title":"Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory","created":"2026-01-31","cited_by_count":16,"area":null},{"id":"2406.19898","title":"Paraphrase Types Elicit Prompt Engineering Capabilities","created":"2024-06-28","cited_by_count":15,"area":null},{"id":"2509.11656","title":"MALLM: Multi-Agent Large Language Models Framework","created":"2025-09-15","cited_by_count":15,"area":"lr"},{"id":"2605.25492","title":"SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks","created":"2026-05-25","cited_by_count":15,"area":null},{"id":"2405.10799","title":"Training Compute Thresholds: Features and Functions in AI Regulation","created":"2024-05-17","cited_by_count":14,"area":null},{"id":"2502.09192","title":"Thinking beyond the anthropomorphic paradigm benefits LLM research","created":"2025-02-13","cited_by_count":14,"area":"lr"},{"id":"2507.10587","title":"Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing","created":"2025-07-11","cited_by_count":14,"area":null},{"id":"2402.12976","title":"The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis","created":"2024-02-20","cited_by_count":13,"area":null},{"id":"2407.15343","title":"Improving Minimum Bayes Risk Decoding with Multi-Prompt","created":"2024-07-22","cited_by_count":13,"area":null},{"id":"2412.02368","title":"ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation?","created":"2024-12-03","cited_by_count":13,"area":"mllm"},{"id":"2503.22353","title":"Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions","created":"2025-03-28","cited_by_count":13,"area":null},{"id":"2504.04893","title":"SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models","created":"2025-04-07","cited_by_count":13,"area":"mllm"},{"id":"2410.18588","title":"Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data","created":"2024-10-24","cited_by_count":12,"area":"synth"},{"id":"2412.17933","title":"BenCzechMark : A Czech-centric Multitask and Multimetric Benchmark for Large Language Models with Duel Scoring Mechanism","created":"2024-12-23","cited_by_count":12,"area":null},{"id":"2601.03746","title":"Whose Facts Win? LLM Source Preferences under Knowledge Conflicts","created":"2026-01-07","cited_by_count":12,"area":"rag"},{"id":"2505.22169","title":"ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments","created":"2025-05-28","cited_by_count":11,"area":null},{"id":"2412.00765","title":"SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts","created":"2024-12-01","cited_by_count":10,"area":"safety"},{"id":"2508.11383","title":"When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs","created":"2025-08-15","cited_by_count":10,"area":null},{"id":"2604.11581","title":"Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking","created":"2026-04-13","cited_by_count":10,"area":null},{"id":"2604.16706","title":"Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents","created":"2026-04-17","cited_by_count":10,"area":"agents"},{"id":"2604.23478","title":"JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems","created":"2026-04-26","cited_by_count":10,"area":null},{"id":"2404.12174","title":"Claim Check-Worthiness Detection: How Well do LLMs Grasp Annotation Guidelines?","created":"2024-04-18","cited_by_count":9,"area":null},{"id":"2410.11672","title":"Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers","created":"2024-10-15","cited_by_count":9,"area":null},{"id":"2510.11905","title":"LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance","created":"2025-10-13","cited_by_count":9,"area":null},{"id":"2512.07795","title":"ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning","created":"2025-12-08","cited_by_count":9,"area":"lr"},{"id":"2406.14117","title":"An Investigation of Prompt Variations for Zero-shot LLM-based Rankers","created":"2024-06-20","cited_by_count":8,"area":null},{"id":"2406.16086","title":"SEAM: A Stochastic Benchmark for Multi-Document Tasks","created":"2024-06-23","cited_by_count":8,"area":null},{"id":"2408.01963","title":"A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios","created":"2024-08-04","cited_by_count":8,"area":null},{"id":"2507.14240","title":"HuggingGraph: Understanding the Supply Chain of LLM Ecosystem","created":"2025-07-17","cited_by_count":8,"area":null},{"id":"2510.07575","title":"Benchmarking is Broken -- Don't Let AI be its Own Judge","created":"2025-10-08","cited_by_count":8,"area":null},{"id":"2601.13537","title":"When Wording Steers the Evaluation: Framing Bias in LLM judges","created":"2026-01-20","cited_by_count":8,"area":null},{"id":"2601.22588","title":"Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry","created":"2026-01-30","cited_by_count":8,"area":"lr"},{"id":"2604.14090","title":"From Weights to Activations: Is Steering the Next Frontier of Adaptation?","created":"2026-04-15","cited_by_count":8,"area":null},{"id":"2604.16403","title":"Computational Hermeneutics: Evaluating generative AI as a cultural technology","created":"2026-03-31","cited_by_count":8,"area":null},{"id":"2605.06161","title":"Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges","created":"2026-05-07","cited_by_count":8,"area":"safety"},{"id":"2605.18890","title":"Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits","created":"2026-05-17","cited_by_count":8,"area":null},{"id":"2605.28873","title":"Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit","created":"2026-05-25","cited_by_count":8,"area":null},{"id":"2402.12486","title":"Do Pre-Trained Language Models Detect and Understand Semantic Underspecification? Ask the DUST!","created":"2024-02-19","cited_by_count":7,"area":null},{"id":"2404.12145","title":"From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency","created":"2024-04-18","cited_by_count":7,"area":null},{"id":"2409.04475","title":"Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluation","created":"2024-09-05","cited_by_count":7,"area":null},{"id":"2409.13860","title":"SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation","created":"2024-09-20","cited_by_count":7,"area":null},{"id":"2501.15000","title":"MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models","created":"2025-01-25","cited_by_count":7,"area":null},{"id":"2502.02289","title":"Evalita-LLM: Benchmarking Large Language Models on Italian","created":"2025-02-04","cited_by_count":7,"area":null},{"id":"2510.05136","title":"Linguistic Characteristics of AI-Generated Text: A Survey","created":"2025-10-01","cited_by_count":7,"area":null},{"id":"2512.14754","title":"Revisiting the Reliability of Language Models in Instruction-Following","created":"2025-12-15","cited_by_count":7,"area":null},{"id":"2602.11328","title":"Evaluating Alignment of Behavioral Dispositions in LLMs","created":"2026-02-11","cited_by_count":7,"area":"alignment"}],"next_cursor":"100"}