mlsanityData to 14 Oct 2026

Rubrik's Cube: Testing a New Rubric for Evaluating Explanations on the CUBE dataset

arXiv 2503.23899 ↗First posted 31 Mar 2025cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

The performance and usability of Large-Language Models (LLMs) are driving their use in explanation generation tasks. However, despite their widespread adoption, LLM explanations have been found to be unreliable, making it difficult for users to distinguish good from bad explanations. To address this issue, we present Rubrik's CUBE, an education-inspired rubric and a dataset of 26k explanations, written and later quality-annotated using the rubric by both humans and six open- and closed-source LLMs. The CUBE dataset focuses on two reasoning and two language tasks, providing the necessary diversity for us to effectively test our proposed rubric. Using Rubrik, we find that explanations are influenced by both task and perceived difficulty. Low quality stems primarily from a lack of conciseness in LLM-generated explanations, rather than cohesion and word choice. The full dataset, rubric, and code are available at https://github.com/RubriksCube/rubriks_cube.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 27 of its 134 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

27 of 134 references.

References · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  5. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  6. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  7. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  8. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  9. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  10. 2023How Language Model Hallucinations Can Snowballcited by 311
  11. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  12. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
15 more
  1. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  2. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 105
  3. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  4. 2020Aligning Faithful Interpretations with their Social Attributioncited by 79
  5. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  6. 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
  7. 2023Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questionscited by 42
  8. 2024Are LLM-based Evaluators Confusing NLG Quality Criteria?cited by 36
  9. 2023Understanding Users' Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Levelcited by 26
  10. 2019Modeling Fluency and Faithfulness for Diverse Neural Machine Translationcited by 13
  11. 2024A review on the use of large language models as virtual tutorscited by 13
  12. 2017A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directionscited by 10
  13. 2024Appropriateness of LLM-equipped Robotic Well-being Coach Language in the Workplace: A Qualitative Evaluationcited by 5
  14. 2024Multi-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Koreancited by 4
  15. 2024ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 63
  2. 2025Reward and Guidance through Rubrics: Promoting Exploration to Improve Multi-Domain Reasoningcited by 21
  3. 2025Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interactioncited by 9
  4. 2026RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoningcited by 5
  5. 2026Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learningcited by 4
  6. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscapecited by 4
  7. 2026Decoupled Physical Modeling and Execution for Physics Reasoningcited by 1
  8. 2026LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generationcited by 0