{"total":52,"rows":[{"id":"2502.14127","title":"Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Above","created":"2025-02-19","cited_by_count":52,"area":null},{"id":"2507.02856","title":"Answer Matching Outperforms Multiple Choice for Language Model Evaluation","created":"2025-07-03","cited_by_count":41,"area":null},{"id":"2407.15018","title":"Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions","created":"2024-07-21","cited_by_count":35,"area":null},{"id":"2503.14996","title":"Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering","created":"2025-03-19","cited_by_count":28,"area":null},{"id":"2410.20245","title":"Improving Model Evaluation using SMART Filtering of Benchmark Datasets","created":"2024-10-26","cited_by_count":21,"area":null},{"id":"2509.18052","title":"The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies","created":"2025-09-22","cited_by_count":14,"area":null},{"id":"2406.02394","title":"Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine","created":"2024-06-04","cited_by_count":13,"area":null},{"id":"2410.15512","title":"Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?","created":"2024-10-20","cited_by_count":13,"area":null},{"id":"2510.09869","title":"NarraBench: A Comprehensive Framework for Narrative Benchmarking","created":"2025-10-10","cited_by_count":13,"area":null},{"id":"2407.01992","title":"Is Your Large Language Model Knowledgeable or a Choices-Only Cheater?","created":"2024-07-02","cited_by_count":12,"area":null},{"id":"2409.18857","title":"Mitigating Selection Bias with Node Pruning and Auxiliary Options","created":"2024-09-27","cited_by_count":12,"area":null},{"id":"2409.00113","title":"Wait, that's not an option: LLMs Robustness with Incorrect Multiple-Choice Options","created":"2024-08-27","cited_by_count":11,"area":null},{"id":"2410.14248","title":"Addressing Blind Guessing: Calibration of Selection Bias in Multiple-Choice Question Answering by Video Language Models","created":"2024-10-18","cited_by_count":11,"area":null},{"id":"2502.18316","title":"WiCkeD: A Simple Method to Make Multiple Choice Benchmarks More Challenging","created":"2025-02-25","cited_by_count":11,"area":null},{"id":"2508.09853","title":"STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports","created":"2025-08-13","cited_by_count":10,"area":null},{"id":"2410.16451","title":"Susu Box or Piggy Bank: Assessing Cultural Commonsense Knowledge between Ghana and the U.S","created":"2024-10-21","cited_by_count":9,"area":null},{"id":"2510.04398","title":"SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations","created":"2025-10-05","cited_by_count":9,"area":null},{"id":"2604.02605","title":"Do Audio-Visual Large Language Models Really See and Hear?","created":"2026-04-03","cited_by_count":9,"area":null},{"id":"2609.37647","title":"Evaluating and Benchmarking the System One Model Jev","created":"2026-09-29","cited_by_count":9,"area":null},{"id":"2411.02018","title":"Shortcut Learning in In-Context Learning: A Survey","created":"2024-11-04","cited_by_count":8,"area":null},{"id":"2510.27055","title":"Detecting Data Contamination in LLMs via In-Context Learning","created":"2025-10-30","cited_by_count":8,"area":null},{"id":"2510.08608","title":"MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation","created":"2025-10-07","cited_by_count":5,"area":null},{"id":"2602.06221","title":"BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2506.00250","title":"PersianMedQA: Evaluating Large Language Models on a Persian-English Bilingual Medical Question Answering Benchmark","created":"2025-05-30","cited_by_count":3,"area":null},{"id":"2508.15239","title":"WangchanThaiInstruct: An instruction-following Dataset for Culture-Aware, Multitask, and Multi-domain Evaluation in Thai","created":"2025-08-21","cited_by_count":3,"area":null},{"id":"2606.23583","title":"Evaluation Awareness Is Not One Capability: Evidence from Open Language Models","created":"2026-06-22","cited_by_count":3,"area":null},{"id":"2410.13029","title":"When Not to Answer: Evaluating Prompts on GPT Models for Effective Abstention in Unanswerable Math Word Problems","created":"2024-10-16","cited_by_count":2,"area":null},{"id":"2502.18798","title":"Choices Speak Louder than Questions","created":"2025-02-26","cited_by_count":2,"area":null},{"id":"2510.05026","title":"Idiom Understanding as a Tool to Measure the Dialect Gap","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2606.11643","title":"Improving Cross-Format Robustness in Language Models with Multi-Format Training","created":"2026-06-10","cited_by_count":2,"area":null},{"id":"2501.04927","title":"Investigating Numerical Translation with Large Language Models","created":"2025-01-09","cited_by_count":1,"area":null},{"id":"2511.21860","title":"Improving Score Reliability of Multiple Choice Benchmarks with Consistency Evaluation and Altered Answer Choices","created":"2025-11-26","cited_by_count":1,"area":null},{"id":"2606.12250","title":"Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?","created":"2026-06-10","cited_by_count":1,"area":null},{"id":"2608.02966","title":"Every Wrong Answer Counts: Option-Level Psychometrics for LLM Multiple-Choice Benchmarks","created":"2026-08-03","cited_by_count":1,"area":null},{"id":"2608.11947","title":"A, B, C, or Stable? Accuracy and Order Sensitivity Diverge in LLM Multiple-Choice Evaluation","created":"2026-08-12","cited_by_count":1,"area":null},{"id":"2608.15428","title":"Gated Against One Model, Open to the Next: Option-Only Solvability in Legal Multiple-Choice Benchmarks","created":"2026-08-15","cited_by_count":1,"area":null},{"id":"2608.30372","title":"Auditing MCQA Benchmarks through Probability Landscapes","created":"2026-08-31","cited_by_count":1,"area":null},{"id":"2506.01275","title":"Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3D","created":"2025-06-02","cited_by_count":0,"area":null},{"id":"2506.12327","title":"Intersectional Bias in Japanese Large Language Models from a Contextualized Perspective","created":"2025-06-14","cited_by_count":0,"area":null},{"id":"2507.23776","title":"Cascaded Information Disclosure for Generalized Evaluation of Problem Solving Capabilities","created":"2025-07-31","cited_by_count":0,"area":null},{"id":"2508.00741","title":"Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data","created":"2025-08-01","cited_by_count":0,"area":null},{"id":"2509.25813","title":"RoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Models","created":"2025-09-30","cited_by_count":0,"area":null},{"id":"2510.00436","title":"Automated Evaluation can Distinguish the Good and Bad AI Responses to Patient Questions about Hospitalization","created":"2025-10-01","cited_by_count":0,"area":null},{"id":"2601.13300","title":"OI-Bench: An Option Injection Benchmark for Evaluating LLM Susceptibility to Directive Interference","created":"2026-01-19","cited_by_count":0,"area":null},{"id":"2602.17377","title":"Corpus Prevalence of Multiple-Choice Question Options","created":"2026-02-19","cited_by_count":0,"area":null},{"id":"2608.21409","title":"Sycophants in the Courtroom: Are LLMs Fragile to Juridical Authority and Evolving Legal Standards?","created":"2026-08-10","cited_by_count":0,"area":null},{"id":"2608.29887","title":"Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation","created":"2026-08-30","cited_by_count":0,"area":null},{"id":"2609.13003","title":"Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage","created":"2026-09-11","cited_by_count":0,"area":null},{"id":"2609.22038","title":"QuranicMMLU: A Cognitively-Aware Benchmark for Evaluating Generative AI Solutions on Quranic Linguistic Knowledge","created":"2026-09-18","cited_by_count":0,"area":null},{"id":"2609.26087","title":"The Architect, the Adversary, and the Judge: Closed-Loop Generation of Standards-Aligned Assessment Items at Scale","created":"2026-08-26","cited_by_count":0,"area":null},{"id":"2609.37494","title":"Your Benchmark Is Not Saturated: Reviving Multiple-Choice Evaluation with Answer Pooling","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2610.05532","title":"DelegationBench: Measuring When AI Agents Should Ask Before Acting","created":"2026-10-04","cited_by_count":0,"area":null}],"next_cursor":null}