# krippendorffsalpha: An R Package for Measuring Agreement Using Krippendorff's Alpha Coefficient

arXiv: https://arxiv.org/abs/2103.12170 · HTML: https://mlsanity.com/paper/2103.12170/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2021.

## Built on by (most cited first, 7 of 7)

- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [A semantics-driven methodology for high-quality image annotation](https://mlsanity.com/paper/2307.14119/) (2023)
- [Incremental Image Labeling via Iterative Refinement](https://mlsanity.com/paper/2304.08989/) (2023)
- [Video-Helpful Multimodal Machine Translation](https://mlsanity.com/paper/2310.20201/) (2023)
- [ReviewScore: Misinformed Peer Review Detection with Large Language Models](https://mlsanity.com/paper/2509.21679/) (2025)
- [When Looks Do Not Lie: Discourse Structure Guided In-Context Learning for Faithful Diagram Generation](https://mlsanity.com/paper/2601.20476/) (2026)
- [Crowdsourcing of Real-world Image Annotation via Visual Properties](https://mlsanity.com/paper/2604.14449/) (2026)

## Abstract (arXiv)

R package krippendorffsalpha provides tools for measuring agreement using Krippendorff's Alpha coefficient, a well-known nonparametric measure of agreement (also called inter-rater reliability and various other names). This article first develops Krippendorff's Alpha in a natural way, and situates Alpha among statistical procedures. Then the usage of package krippendorffsalpha is illustrated via analyses of two datasets, the latter of which was collected during an imaging study of hip cartilage. The package permits users to apply the Alpha methodology using built-in distance functions for the nominal, ordinal, interval, or ratio levels of measurement. User-defined distance functions are also supported. The fitting function can accommodate any number of units, any number of coders, and missingness. Bootstrap inference is supported, and the bootstrap computation can be carried out in parallel.