mlsanityData to 7 Oct 2026

CRACQ: A Multi-Dimensional Approach To Automated Document Assessment

arXiv 2510.02337 ↗First posted 26 Sep 2025cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents CRACQ, a multi-dimensional evaluation framework tailored to evaluate documents across f i v e specific traits: Coherence, Rigor, Appropriateness, Completeness, and Quality. Building on insights from traitbased Automated Essay Scoring (AES), CRACQ expands its fo-cus beyond essays to encompass diverse forms of machine-generated text, providing a rubricdriven and interpretable methodology for automated evaluation. Unlike singlescore approaches, CRACQ integrates linguistic, semantic, and structural signals into a cumulative assessment, enabling both holistic and trait-level analysis. Trained on 500 synthetic grant pro-posals, CRACQ was benchmarked against an LLM-as-a-judge and further tested on both strong and weak real applications. Preliminary results in-dicate that CRACQ produces more stable and interpretable trait-level judgments than direct LLM evaluation, though challenges in reliability and domain scope remain

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

15 of 18 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  4. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  5. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  6. 2024A Survey on LoRA of Large Language Modelscited by 67
  7. 2024How to set AdamW's weight decay as you scale model and dataset sizecited by 42
  8. 2020GRUEN for Evaluating Linguistic Quality of Generated Textcited by 37
  9. 2021Generalized Huber Loss for Robust Learning and its Efficient Minimization for a Robust Statisticscited by 32
  10. 2025Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubricscited by 29
  11. 2023Classifier Calibration with ROC-Regularized Isotonic Regressioncited by 19
  12. 2025TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoringcited by 12
3 more
  1. 2025Training an LLM-as-a-Judge Model: Pipeline, Insights, and Practical Lessonscited by 9
  2. 2025A Judge-free LLM Open-ended Generation Benchmark Based on the Distributional Hypothesiscited by 4
  3. 2019Criteria for assessing grant applications: A systematic reviewcited by 1

Built on by

Not cited yet.