# Operational Collective Intelligence of Humans and Machines

arXiv: https://arxiv.org/abs/2402.13273 · HTML: https://mlsanity.com/paper/2402.13273/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 57 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017)
- [Hybrid Intelligence](https://mlsanity.com/paper/2105.00691/) (2021)
- [Forecast combinations: an over 50-year review](https://mlsanity.com/paper/2205.04216/) (2022)
- [Automated Detection and Forecasting of COVID-19 using Deep Learning Techniques: A Review](https://mlsanity.com/paper/2007.10785/) (2020)
- [Cheaper and Better: Selecting Good Workers for Crowdsourcing](https://mlsanity.com/paper/1502.00725/) (2015)
- [Revisiting Citizen Science Through the Lens of Hybrid Intelligence](https://mlsanity.com/paper/2104.14961/) (2021)
- [Toward a Research Agenda in Adversarial Reasoning: Computational Approaches to Anticipating the Opponent's Intent and Actions](https://mlsanity.com/paper/1512.07943/) (2015)
- [Comparing Psychometric and Behavioral Predictors of Compliance During Human-AI Interactions](https://mlsanity.com/paper/2302.01854/) (2023)

## Abstract (arXiv)

We explore the use of aggregative crowdsourced forecasting (ACF) as a mechanism to help operationalize ``collective intelligence'' of human-machine teams for coordinated actions. We adopt the definition for Collective Intelligence as: ``A property of groups that emerges from synergies among data-information-knowledge, software-hardware, and individuals (those with new insights as well as recognized authorities) that enables just-in-time knowledge for better decisions than these three elements acting alone.'' Collective Intelligence emerges from new ways of connecting humans and AI to enable decision-advantage, in part by creating and leveraging additional sources of information that might otherwise not be included. Aggregative crowdsourced forecasting (ACF) is a recent key advancement towards Collective Intelligence wherein predictions (X% probability that Y will happen) and rationales (why I believe it is this probability that X will happen) are elicited independently from a diverse crowd, aggregated, and then used to inform higher-level decision-making. This research asks whether ACF, as a key way to enable Operational Collective Intelligence, could be brought to bear on operational scenarios (i.e., sequences of events with defined agents, components, and interactions) and decision-making, and considers whether such a capability could provide novel operational capabilities to enable new forms of decision-advantage.