{"total":170,"rows":[{"id":"2310.16789","title":"Detecting Pretraining Data from Large Language Models","created":"2023-10-25","cited_by_count":387,"area":null},{"id":"2310.18018","title":"NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark","created":"2023-10-27","cited_by_count":348,"area":null},{"id":"2310.17623","title":"Proving Test Set Contamination in Black Box Language Models","created":"2023-10-26","cited_by_count":230,"area":null},{"id":"2402.03927","title":"Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs","created":"2024-02-06","cited_by_count":229,"area":null},{"id":"2406.04244","title":"Benchmark Data Contamination of Large Language Models: A Survey","created":"2024-06-06","cited_by_count":147,"area":null},{"id":"2305.10160","title":"Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks","created":"2023-05-17","cited_by_count":142,"area":null},{"id":"2402.15938","title":"Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models","created":"2024-02-24","cited_by_count":122,"area":null},{"id":"2310.03668","title":"GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction","created":"2023-10-05","cited_by_count":119,"area":null},{"id":"2409.19839","title":"ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities","created":"2024-09-30","cited_by_count":110,"area":null},{"id":"2402.18060","title":"Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions","created":"2024-02-28","cited_by_count":109,"area":null},{"id":"2404.18824","title":"Benchmarking Benchmark Leakage in Large Language Models","created":"2024-04-29","cited_by_count":106,"area":null},{"id":"2312.16337","title":"Task Contamination: Language Models May Not Be Few-Shot Anymore","created":"2023-12-26","cited_by_count":99,"area":null},{"id":"2407.14985","title":"Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data","created":"2024-07-20","cited_by_count":85,"area":null},{"id":"2304.00612","title":"Eight Things to Know about Large Language Models","created":"2023-04-02","cited_by_count":73,"area":null},{"id":"2402.19379","title":"Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy","created":"2024-02-29","cited_by_count":67,"area":null},{"id":"2403.04811","title":"Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models","created":"2024-03-06","cited_by_count":65,"area":null},{"id":"2603.21687","title":"MIRAGE: The Illusion of Visual Understanding","created":"2026-03-23","cited_by_count":65,"area":null},{"id":"2210.03588","title":"Understanding Transformer Memorization Recall Through Idioms","created":"2022-10-07","cited_by_count":62,"area":null},{"id":"2502.06559","title":"Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation","created":"2025-02-10","cited_by_count":60,"area":null},{"id":"2502.14127","title":"Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Above","created":"2025-02-19","cited_by_count":52,"area":null},{"id":"2408.02784","title":"LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theory","created":"2024-08-05","cited_by_count":47,"area":null},{"id":"2412.13670","title":"AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge","created":"2024-12-18","cited_by_count":44,"area":null},{"id":"2306.07075","title":"Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence","created":"2023-06-12","cited_by_count":42,"area":null},{"id":"2308.15474","title":"A General-Purpose Self-Supervised Model for Computational Pathology","created":"2023-08-29","cited_by_count":42,"area":null},{"id":"2310.10628","title":"Data Contamination Through the Lens of Time","created":"2023-10-16","cited_by_count":40,"area":null},{"id":"2211.07830","title":"Prompting Language Models for Linguistic Structure","created":"2022-11-15","cited_by_count":39,"area":null},{"id":"2402.16795","title":"If in a Crowdsourced Data Annotation Pipeline, a GPT-4","created":"2024-02-26","cited_by_count":38,"area":null},{"id":"2502.17521","title":"Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation","created":"2025-02-23","cited_by_count":38,"area":null},{"id":"2310.09266","title":"User Inference Attacks on Large Language Models","created":"2023-10-13","cited_by_count":36,"area":null},{"id":"2404.06209","title":"Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models","created":"2024-04-09","cited_by_count":36,"area":null},{"id":"2407.07890","title":"Training on the Test Task Confounds Evaluation and Emergence","created":"2024-07-10","cited_by_count":34,"area":null},{"id":"2206.09860","title":"Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias","created":"2022-06-20","cited_by_count":33,"area":null},{"id":"2310.19619","title":"Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models","created":"2023-10-30","cited_by_count":32,"area":null},{"id":"2405.13769","title":"Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation","created":"2024-05-22","cited_by_count":32,"area":null},{"id":"2403.02990","title":"Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges","created":"2024-03-05","cited_by_count":30,"area":null},{"id":"2410.14826","title":"SPRIG: Improving Large Language Model Performance by System Prompt Optimization","created":"2024-10-18","cited_by_count":29,"area":null},{"id":"2405.01719","title":"Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks","created":"2024-05-02","cited_by_count":28,"area":null},{"id":"2309.17002","title":"Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks","created":"2023-09-29","cited_by_count":27,"area":null},{"id":"2310.12902","title":"Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling","created":"2023-10-19","cited_by_count":27,"area":null},{"id":"2310.11053","title":"Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learning","created":"2023-10-17","cited_by_count":26,"area":null},{"id":"2401.12192","title":"Text Embedding Inversion Security for Multilingual Language Models","created":"2024-01-22","cited_by_count":26,"area":null},{"id":"2410.18966","title":"Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions","created":"2024-10-24","cited_by_count":26,"area":null},{"id":"2308.01285","title":"Flows: Building Blocks of Reasoning and Collaborating AI","created":"2023-08-02","cited_by_count":25,"area":null},{"id":"2405.02743","title":"Beyond Performance: Quantifying and Mitigating Label Bias in LLMs","created":"2024-05-04","cited_by_count":25,"area":null},{"id":"2406.13069","title":"Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG","created":"2024-06-18","cited_by_count":25,"area":null},{"id":"2410.22257","title":"FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation","created":"2024-10-29","cited_by_count":25,"area":null},{"id":"2310.13014","title":"Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament","created":"2023-10-17","cited_by_count":24,"area":null},{"id":"2405.11930","title":"Data Contamination Calibration for Black-box LLMs","created":"2024-05-20","cited_by_count":24,"area":null},{"id":"2310.16379","title":"Evaluating General-Purpose AI with Psychometrics","created":"2023-10-25","cited_by_count":23,"area":null},{"id":"2502.21098","title":"Re-evaluating Theory of Mind evaluation in large language models","created":"2025-02-28","cited_by_count":23,"area":null},{"id":"2301.10416","title":"AI vs. Human -- Differentiation Analysis of Scientific Content Generation","created":"2023-01-24","cited_by_count":21,"area":null},{"id":"2402.07862","title":"AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy","created":"2024-02-12","cited_by_count":21,"area":null},{"id":"2410.20245","title":"Improving Model Evaluation using SMART Filtering of Benchmark Datasets","created":"2024-10-26","cited_by_count":21,"area":null},{"id":"2308.07120","title":"Position: Key Claims in LLM Research Have a Long Tail of Footnotes","created":"2023-08-14","cited_by_count":20,"area":null},{"id":"2407.21530","title":"Data Contamination Report from the 2024 CONDA Shared Task","created":"2024-07-31","cited_by_count":20,"area":null},{"id":"2501.07493","title":"Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards","created":"2025-01-13","cited_by_count":20,"area":null},{"id":"2303.14770","title":"Koala: An Index for Quantifying Overlaps with Pre-training Corpora","created":"2023-03-26","cited_by_count":19,"area":null},{"id":"2310.08256","title":"Impact of Co-occurrence on Factual Knowledge of Large Language Models","created":"2023-10-12","cited_by_count":19,"area":null},{"id":"2410.01769","title":"Quantifying Generalization Complexity for Large Language Models","created":"2024-10-02","cited_by_count":18,"area":null},{"id":"2402.18031","title":"Corpus-Steered Query Expansion with Large Language Models","created":"2024-02-28","cited_by_count":17,"area":null},{"id":"2409.09467","title":"Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI","created":"2024-09-14","cited_by_count":17,"area":null},{"id":"2305.04812","title":"Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns","created":"2023-05-08","cited_by_count":16,"area":null},{"id":"2409.16118","title":"TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models","created":"2024-09-24","cited_by_count":16,"area":null},{"id":"2412.06745","title":"ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities","created":"2024-12-09","cited_by_count":15,"area":null},{"id":"2502.00678","title":"How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergence","created":"2025-02-02","cited_by_count":15,"area":null},{"id":"2402.01656","title":"Promises and pitfalls of artificial intelligence for legal applications","created":"2024-01-10","cited_by_count":14,"area":null},{"id":"2406.03181","title":"Missci: Reconstructing Fallacies in Misrepresented Science","created":"2024-06-05","cited_by_count":14,"area":null},{"id":"2410.03249","title":"How Much Can We Forget about Data Contamination?","created":"2024-10-04","cited_by_count":14,"area":null},{"id":"2405.19740","title":"PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations","created":"2024-05-30","cited_by_count":13,"area":null},{"id":"2404.00699","title":"A Comprehensive Survey of Contamination Detection Methods in Large Language Models","created":"2024-03-31","cited_by_count":12,"area":null},{"id":"2410.16848","title":"ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage","created":"2024-10-22","cited_by_count":12,"area":null},{"id":"2503.13572","title":"VeriContaminated: Assessing LLM-Driven Verilog Coding for Data Contamination","created":"2025-03-17","cited_by_count":12,"area":null},{"id":"2602.10367","title":"LiveMedBench: A Contamination-Free Medical Benchmark for LLMs with Automated Rubric Evaluation","created":"2026-02-10","cited_by_count":12,"area":null},{"id":"2212.10755","title":"JASMINE: Arabic GPT Models for Few-Shot Learning","created":"2022-12-21","cited_by_count":11,"area":null},{"id":"2306.00168","title":"Measuring the Robustness of NLP Models to Domain Shifts","created":"2023-05-31","cited_by_count":11,"area":null},{"id":"2407.21248","title":"Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens","created":"2024-07-30","cited_by_count":11,"area":null},{"id":"2402.19472","title":"Efficient Lifelong Model Evaluation in an Era of Rapid Progress","created":"2024-02-29","cited_by_count":10,"area":null},{"id":"2405.15523","title":"The Mosaic Memory of Large Language Models","created":"2024-05-24","cited_by_count":10,"area":null},{"id":"2507.11405","title":"DCR: Quantifying Data Contamination in LLMs Evaluation","created":"2025-07-15","cited_by_count":10,"area":null},{"id":"2509.24789","title":"Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting","created":"2025-09-29","cited_by_count":10,"area":null},{"id":"2310.15113","title":"Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model","created":"2023-10-23","cited_by_count":9,"area":null},{"id":"2311.09006","title":"Data Similarity is Not Enough to Explain Language Model Performance","created":"2023-11-15","cited_by_count":9,"area":null},{"id":"2406.12809","title":"Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?","created":"2024-06-18","cited_by_count":9,"area":null},{"id":"2408.03281","title":"StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation","created":"2024-08-06","cited_by_count":9,"area":null},{"id":"2410.12029","title":"On Classification with Large Language Models in Cultural Analytics","created":"2024-10-15","cited_by_count":9,"area":null},{"id":"2505.22251","title":"Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition","created":"2025-05-28","cited_by_count":9,"area":null},{"id":"2403.06869","title":"Impact of Noisy Supervision in Foundation Model Learning","created":"2024-03-11","cited_by_count":8,"area":null},{"id":"2406.11380","title":"Evaluating LLMs for Quotation Attribution in Literary Texts: A Case Study of LLaMa3","created":"2024-06-17","cited_by_count":8,"area":null},{"id":"2410.10880","title":"Fine-tuning can Help Detect Pretraining Data from Large Language Models","created":"2024-10-09","cited_by_count":8,"area":null},{"id":"2502.15919","title":"Mind the Gap! Static and Interactive Evaluations of Large Audio Models","created":"2025-02-21","cited_by_count":8,"area":null},{"id":"2504.13416","title":"STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings","created":"2025-04-18","cited_by_count":8,"area":null},{"id":"2505.00612","title":"Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation","created":"2025-05-01","cited_by_count":8,"area":null},{"id":"2510.07575","title":"Benchmarking is Broken -- Don't Let AI be its Own Judge","created":"2025-10-08","cited_by_count":8,"area":null},{"id":"2602.12413","title":"Soft Contamination Means Benchmarks Test Shallow Generalization","created":"2026-02-12","cited_by_count":8,"area":null},{"id":"2307.12418","title":"HateModerate: Testing Hate Speech Detectors against Content Moderation Policies","created":"2023-07-23","cited_by_count":7,"area":null},{"id":"2505.18102","title":"CapBencher: Give Your LLM Benchmark a Built-in Alarm for Test-Set Overfitting","created":"2025-05-23","cited_by_count":7,"area":null},{"id":"2508.17416","title":"Data Leakage in Visual Datasets","created":"2025-08-24","cited_by_count":7,"area":null},{"id":"2310.10062","title":"A Comprehensive Evaluation of Tool-Assisted Generation Strategies","created":"2023-10-16","cited_by_count":6,"area":null},{"id":"2310.10190","title":"Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparison","created":"2023-10-16","cited_by_count":6,"area":null},{"id":"2404.08656","title":"Linear Cross-document Event Coreference Resolution with X-AMR","created":"2024-03-25","cited_by_count":6,"area":null}],"next_cursor":"100"}