# It's just a matter of perspective(s): Crowd-Powered Consensus Organization of Corpora

arXiv: https://arxiv.org/abs/1601.02034 · HTML: https://mlsanity.com/paper/1601.02034/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2016.

## Built on by (most cited first, 1 of 1)

- [Revisiting Prompt Engineering via Declarative Crowdsourcing](https://mlsanity.com/paper/2308.03854/) (2023)

## Abstract (arXiv)

We study the problem of organizing a collection of objects - images, videos - into clusters, using crowdsourcing. This problem is notoriously hard for computers to do automatically, and even with crowd workers, is challenging to orchestrate: (a) workers may cluster based on different latent hierarchies or perspectives; (b) workers may cluster at different granularities even when clustering using the same perspective; and (c) workers may only see a small portion of the objects when deciding how to cluster them (and therefore have limited understanding of the "big picture"). We develop cost-efficient, accurate algorithms for identifying the consensus organization (i.e., the organizing perspective most workers prefer to employ), and incorporate these algorithms into a cost-effective workflow for organizing a collection of objects, termed ORCHESTRA. We compare our algorithms with other algorithms for clustering, on a variety of real-world datasets, and demonstrate that ORCHESTRA organizes items better and at significantly lower costs.