Scorio.jl: A Julia package for ranking stochastic responses
Not on the map. Why?
Abstract (arXiv)
Scorio.jl is a Julia package for evaluating and ranking systems from repeated responses to shared tasks. It provides a common tensor-based interface for direct score-based, pairwise, psychometric, voting, graph, and listwise methods, so the same benchmark can be analyzed under multiple ranking assumptions. We describe the package design, position it relative to existing Julia tools, and report pilot experiments on synthetic rank recovery, stability under limited trials, and runtime scaling.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 7 of its 42 references that are in the map. First posted on arXiv 14 Mar 2026.
Builds on
7 of 42 references.
References · 7
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2019α-Rank: Multi-Agent Evaluation by Evolutioncited by 50
- 2018Re-evaluating Evaluationcited by 47
- 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
- 2014Spectral Ranking using Seriationcited by 9
- 2019Davidson-Luce model for multi-item choice with tiescited by 5
- 2026Ranking Reasoning LLMs under Test-Time Scalingcited by 1
Built on by
Not cited yet.