{"total":162,"rows":[{"id":"2412.05579","title":"LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods","created":"2024-12-07","cited_by_count":465,"area":null},{"id":"2501.10970","title":"The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs","created":"2025-01-19","cited_by_count":74,"area":null},{"id":"2410.13341","title":"Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data","created":"2024-10-17","cited_by_count":51,"area":null},{"id":"2408.05534","title":"Can LLMs Replace Manual Annotation of Software Engineering Artifacts?","created":"2024-08-10","cited_by_count":42,"area":null},{"id":"2508.18076","title":"Neither Valid nor Reliable? Investigating the Use of LLMs as Judges","created":"2025-08-25","cited_by_count":39,"area":null},{"id":"2510.27106","title":"Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworks","created":"2025-10-31","cited_by_count":38,"area":null},{"id":"2408.08781","title":"Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions","created":"2024-08-16","cited_by_count":34,"area":null},{"id":"2410.03775","title":"Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge","created":"2024-10-03","cited_by_count":31,"area":null},{"id":"2412.09569","title":"JuStRank: Benchmarking LLM Judges for System Ranking","created":"2024-12-12","cited_by_count":29,"area":null},{"id":"2606.19544","title":"Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias","created":"2026-06-17","cited_by_count":28,"area":null},{"id":"2510.09738","title":"Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement","created":"2025-10-10","cited_by_count":25,"area":null},{"id":"2411.02316","title":"Evaluating Creative Short Story Generation in Humans and Large Language Models","created":"2024-11-04","cited_by_count":21,"area":null},{"id":"2504.13677","title":"Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results","created":"2025-04-18","cited_by_count":21,"area":null},{"id":"2410.18889","title":"Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance","created":"2024-10-24","cited_by_count":20,"area":null},{"id":"2504.10063","title":"Hallucination Detection in LLMs with Topological Divergence on Attention Graphs","created":"2025-04-14","cited_by_count":19,"area":null},{"id":"2410.11594","title":"Black-box Uncertainty Quantification Method for LLM-as-a-Judge","created":"2024-10-15","cited_by_count":18,"area":null},{"id":"2505.16222","title":"Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation","created":"2025-05-22","cited_by_count":18,"area":null},{"id":"2602.00521","title":"Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory","created":"2026-01-31","cited_by_count":16,"area":null},{"id":"2502.01523","title":"CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering","created":"2025-02-03","cited_by_count":15,"area":null},{"id":"2412.14140","title":"GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking","created":"2024-12-18","cited_by_count":14,"area":null},{"id":"2507.10587","title":"Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing","created":"2025-07-11","cited_by_count":14,"area":null},{"id":"2602.09383","title":"BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation","created":"2026-02-10","cited_by_count":14,"area":null},{"id":"2502.04362","title":"LLMs can be easily Confused by Instructional Distractions","created":"2025-02-05","cited_by_count":13,"area":null},{"id":"2502.07445","title":"Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleon","created":"2025-02-11","cited_by_count":13,"area":null},{"id":"2504.09714","title":"Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish","created":"2025-04-13","cited_by_count":13,"area":null},{"id":"2411.17637","title":"On Limitations of LLM as Annotator for Low Resource Languages","created":"2024-11-26","cited_by_count":12,"area":null},{"id":"2411.05980","title":"FactLens: Benchmarking Fine-Grained Fact Verification","created":"2024-11-08","cited_by_count":10,"area":null},{"id":"2506.19571","title":"Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress","created":"2025-06-24","cited_by_count":10,"area":null},{"id":"2607.01233","title":"Measuring the Gap Between Human and LLM Research Ideas","created":"2026-07-01","cited_by_count":10,"area":null},{"id":"2504.07971","title":"SPHERE: An Evaluation Card for Human-AI Systems","created":"2025-03-24","cited_by_count":9,"area":null},{"id":"2507.17015","title":"Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?","created":"2025-07-22","cited_by_count":9,"area":null},{"id":"2603.12520","title":"When LLM Judge Scores Look Good but Best-of-N Decisions Fail","created":"2026-03-12","cited_by_count":9,"area":null},{"id":"2505.23799","title":"Estimating LLM Consistency: A User Baseline vs Surrogate Metrics","created":"2025-05-26","cited_by_count":8,"area":null},{"id":"2601.13537","title":"When Wording Steers the Evaluation: Framing Bias in LLM judges","created":"2026-01-20","cited_by_count":8,"area":null},{"id":"2504.08697","title":"LLMs as Span Annotators: A Comparative Study of LLMs and Humans","created":"2025-04-11","cited_by_count":7,"area":null},{"id":"2507.02870","title":"Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models","created":"2025-06-06","cited_by_count":7,"area":null},{"id":"2507.03772","title":"Skewed Score: A statistical framework to assess autograders","created":"2025-07-04","cited_by_count":7,"area":null},{"id":"2601.08654","title":"From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges","created":"2026-01-13","cited_by_count":7,"area":null},{"id":"2601.21817","title":"A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth","created":"2026-01-29","cited_by_count":7,"area":null},{"id":"2503.23243","title":"Evaluating how LLM annotations represent diverse views on contentious topics","created":"2025-03-29","cited_by_count":6,"area":null},{"id":"2506.02945","title":"Quantitative LLM Judges","created":"2025-06-03","cited_by_count":6,"area":null},{"id":"2507.20301","title":"Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation","created":"2025-07-27","cited_by_count":6,"area":null},{"id":"2508.10175","title":"Estimating Machine Translation Difficulty","created":"2025-08-13","cited_by_count":6,"area":null},{"id":"2606.00093","title":"Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Why","created":"2026-05-25","cited_by_count":6,"area":null},{"id":"2609.29769","title":"Decision Models vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places","created":"2026-09-24","cited_by_count":6,"area":null},{"id":"2504.16921","title":"IberBench: LLM Evaluation on Iberian Languages","created":"2025-04-23","cited_by_count":5,"area":null},{"id":"2505.16134","title":"Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMs","created":"2025-05-22","cited_by_count":5,"area":null},{"id":"2505.21364","title":"Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders","created":"2025-05-27","cited_by_count":5,"area":null},{"id":"2507.01936","title":"The Thin Line Between Comprehension and Persuasion in LLMs","created":"2025-07-02","cited_by_count":5,"area":null},{"id":"2509.22703","title":"AccessEval: Benchmarking Disability Bias in Large Language Models","created":"2025-09-22","cited_by_count":5,"area":null},{"id":"2511.05124","title":"QuAnTS: Question Answering on Time Series","created":"2025-11-07","cited_by_count":5,"area":null},{"id":"2602.06221","title":"BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2408.09777","title":"Summarizing long regulatory documents with a multi-step pipeline","created":"2024-08-19","cited_by_count":4,"area":null},{"id":"2409.07131","title":"Reranking Laws for Language Generation: A Communication-Theoretic Perspective","created":"2024-09-11","cited_by_count":4,"area":null},{"id":"2504.15815","title":"What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns","created":"2025-04-22","cited_by_count":4,"area":null},{"id":"2505.04209","title":"To Judge or not to Judge: Using LLM Judgements for Advertiser Keyphrase Relevance at eBay","created":"2025-05-07","cited_by_count":4,"area":null},{"id":"2505.05445","title":"clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisations","created":"2025-05-08","cited_by_count":4,"area":null},{"id":"2506.17111","title":"Are Bias Evaluation Methods Biased ?","created":"2025-06-20","cited_by_count":4,"area":null},{"id":"2506.23929","title":"IMPACT: Inflectional Morphology Probes Across Complex Typologies","created":"2025-06-30","cited_by_count":4,"area":null},{"id":"2507.15557","title":"Evaluating Text Style Transfer: A Nine-Language Benchmark for Text Detoxification","created":"2025-07-21","cited_by_count":4,"area":null},{"id":"2508.20186","title":"AI Propaganda factories with language models","created":"2025-08-27","cited_by_count":4,"area":null},{"id":"2510.08111","title":"Evaluating LLM-Generated Legal Explanations for Regulatory Compliance in Social Media Influencer Marketing","created":"2025-10-09","cited_by_count":4,"area":null},{"id":"2511.07017","title":"Benchmarking LLMs for Fine-Grained Code Review with Enriched Context in Practice","created":"2025-11-10","cited_by_count":4,"area":null},{"id":"2604.09497","title":"BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluation","created":"2026-04-10","cited_by_count":4,"area":null},{"id":"2605.30803","title":"PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges","created":"2026-05-29","cited_by_count":4,"area":null},{"id":"2502.18339","title":"Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks","created":"2025-02-24","cited_by_count":3,"area":null},{"id":"2506.06775","title":"They want to pretend not to understand: The Limits of Current LLMs in Interpreting Implicit Content of Political Discourse","created":"2025-06-07","cited_by_count":3,"area":null},{"id":"2506.09147","title":"LLM-as-a-qualitative-judge: automating error analysis in natural language generation","created":"2025-06-10","cited_by_count":3,"area":null},{"id":"2507.06774","title":"Checklist Engineering Empowers Multilingual LLM Judges","created":"2025-07-09","cited_by_count":3,"area":null},{"id":"2507.21130","title":"INTEGRALBENCH: Benchmarking LLMs with Definite Integral Problems","created":"2025-07-22","cited_by_count":3,"area":null},{"id":"2508.17164","title":"The Impact of Annotator Personas on LLM Behavior Across the Perspectivism Spectrum","created":"2025-08-23","cited_by_count":3,"area":null},{"id":"2509.14899","title":"CARGO: A Framework for Confidence-Aware Routing of Large Language Models","created":"2025-09-18","cited_by_count":3,"area":null},{"id":"2511.18597","title":"Toward Trustworthy Difficulty Assessments: Large Language Models as Judges in Programming and Synthetic Tasks","created":"2025-11-23","cited_by_count":3,"area":null},{"id":"2601.08064","title":"Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations","created":"2026-01-12","cited_by_count":3,"area":null},{"id":"2603.18557","title":"Cross-Lingual LLM-Judge Transfer via Evaluation Decomposition","created":"2026-03-19","cited_by_count":3,"area":null},{"id":"2604.05083","title":"Beyond LLM-as-a-Judge: Deterministic Metrics for Multilingual Generative Text Evaluation","created":"2026-04-06","cited_by_count":3,"area":null},{"id":"2605.01771","title":"The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't","created":"2026-05-03","cited_by_count":3,"area":null},{"id":"2607.13347","title":"LLM-as-a-Judge Scores Are Unreliable Optimization Signals in Closed-Loop Table Recognition","created":"2026-07-15","cited_by_count":3,"area":null},{"id":"2504.11934","title":"An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation","created":"2025-04-16","cited_by_count":2,"area":null},{"id":"2505.15087","title":"HopWeaver: Cross-Document Synthesis of High-Quality and Authentic Multi-Hop Questions","created":"2025-05-21","cited_by_count":2,"area":null},{"id":"2506.14335","title":"References Matter: Investigating the Impact of Reference Set Variation on Summarization Evaluation","created":"2025-06-17","cited_by_count":2,"area":null},{"id":"2509.26302","title":"QUARTZ : QA-based Unsupervised Abstractive Refinement for Task-oriented Dialogue Summarization","created":"2025-09-30","cited_by_count":2,"area":null},{"id":"2510.20700","title":"Structure-Conditional Minimum Bayes Risk Decoding","created":"2025-10-23","cited_by_count":2,"area":null},{"id":"2512.06751","title":"Becoming Experienced Judges: Selective Test-Time Learning for Evaluators","created":"2025-12-07","cited_by_count":2,"area":null},{"id":"2512.19475","title":"A Large-Language-Model Framework for Automated Humanitarian Situation Reporting","created":"2025-12-22","cited_by_count":2,"area":null},{"id":"2603.26539","title":"How Open Must Language Models be to Enable Reliable Scientific Inference?","created":"2026-03-27","cited_by_count":2,"area":null},{"id":"2604.03695","title":"POEMetric: The Last Stanza of Humanity","created":"2026-04-04","cited_by_count":2,"area":null},{"id":"2604.17316","title":"Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA","created":"2026-04-19","cited_by_count":2,"area":null},{"id":"2606.07726","title":"Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm that Provably Exploits Model Similarity","created":"2026-06-05","cited_by_count":2,"area":null},{"id":"2606.11643","title":"Improving Cross-Format Robustness in Language Models with Multi-Format Training","created":"2026-06-10","cited_by_count":2,"area":null},{"id":"2606.11762","title":"Automated Creativity Evaluation of Language Models Across Open-Ended Tasks","created":"2026-06-10","cited_by_count":2,"area":null},{"id":"2410.10665","title":"Double Jeopardy and Climate Impact in the Use of Large Language Models: Socio-economic Disparities and Reduced Utility for Non-English Speakers","created":"2024-10-14","cited_by_count":1,"area":null},{"id":"2410.11265","title":"In-Context Learning for Long-Context Sentiment Analysis on Infrastructure Project Opinions","created":"2024-10-15","cited_by_count":1,"area":null},{"id":"2410.19812","title":"Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models","created":"2024-10-17","cited_by_count":1,"area":null},{"id":"2502.04997","title":"Aligning Black-box Language Models with Human Judgments","created":"2025-02-07","cited_by_count":1,"area":null},{"id":"2502.12052","title":"A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better Interpretability","created":"2025-02-17","cited_by_count":1,"area":null},{"id":"2502.13034","title":"Natural Language Generation from Visual Events: State-of-the-Art and Key Open Questions","created":"2025-02-18","cited_by_count":1,"area":null},{"id":"2502.20635","title":"Can LLM Assist in the Evaluation of the Quality of Machine Learning Explanations?","created":"2025-02-28","cited_by_count":1,"area":null},{"id":"2503.00619","title":"PinLanding: Content-First Keyword Landing Page Generation via Multi-Modal AI for Web-Scale Discovery","created":"2025-03-01","cited_by_count":1,"area":null},{"id":"2504.07174","title":"HypoEval: Hypothesis-Guided Evaluation for Natural Language Generation","created":"2025-04-09","cited_by_count":1,"area":null}],"next_cursor":"100"}