mlsanityData to 14 Oct 2026

How Model Size, Temperature, and Prompt Style Affect LLM-Human Assessment Score Alignment

arXiv 2509.19329 ↗First posted 14 Sep 2025cs.CL stat.MEcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We examined how model size, temperature, and prompt style affect Large Language Models' (LLMs) alignment within itself, between models, and with human in assessing clinical reasoning skills. Model size emerged as a key factor in LLM-human score alignment. Study highlights the importance of checking alignments across multiple levels.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 1 of its 23 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

1 of 23 references.

References · 1

  1. 2024Can AI grade your essays? A comparative analysis of large language models and teacher ratings in multidimensional essay scoringcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Affording Process Auditability with QualAnalyzer: An Atomistic LLM Analysis Tool for Qualitative Researchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale
  2. 2024Dissecting Human and LLM Preferences
  3. 2026Evaluating LLMs as Human Surrogates in Controlled Experiments
  4. 2026Exploring the Effects of Alignment on Numerical Bias in Large Language Models
  5. 2025Can Finetuing LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence?
  6. 2026Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judge
  7. 2024Understanding The Effect Of Temperature On Alignment With Human Opinions
  8. 2025Scaling Item-to-Standard Alignment with Large Language Models: Accuracy, Limits, and Solutions