# Water from Two Rocks: Maximizing the Mutual Information

arXiv: https://arxiv.org/abs/1802.08887 · HTML: https://mlsanity.com/paper/1802.08887/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2018.

## Built on by (most cited first, 10 of 10)

- [Benchmarking LLMs' Judgments with No Gold Standard](https://mlsanity.com/paper/2411.07127/) (2024)
- [Data Distribution Valuation](https://mlsanity.com/paper/2410.04386/) (2024)
- [Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms](https://mlsanity.com/paper/2402.13567/) (2024)
- [Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization](https://mlsanity.com/paper/2204.02485/) (2022)
- [Truthful Generalized Linear Models](https://mlsanity.com/paper/2209.07815/) (2022)
- [A Cram'er-von Mises Approach to Incentivizing Truthful Data Sharing](https://mlsanity.com/paper/2506.07272/) (2025)
- [Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes](https://mlsanity.com/paper/2508.05469/) (2025)
- [PMIScore: An Unsupervised Approach to Quantify Dialogue Engagement](https://mlsanity.com/paper/2603.13796/) (2026)
- [Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics](https://mlsanity.com/paper/2608.01423/) (2026)
- [How the Audit Rule Shapes Faithful Factor Explanations in LLMs](https://mlsanity.com/paper/2610.01514/) (2026)

## Abstract (arXiv)

We build a natural connection between the learning problem, co-training, and forecast elicitation without verification (related to peer-prediction) and address them simultaneously using the same information theoretic approach. In co-training/multiview learning, the goal is to aggregate two views of data into a prediction for a latent label. We show how to optimally combine two views of data by reducing the problem to an optimization problem. Our work gives a unified and rigorous approach to the general setting. In forecast elicitation without verification we seek to design a mechanism that elicits high quality forecasts from agents in the setting where the mechanism does not have access to the ground truth. By assuming the agents' information is independent conditioning on the outcome, we propose mechanisms where truth-telling is a strict equilibrium for both the single-task and multi-task settings. Our multi-task mechanism additionally has the property that the truth-telling equilibrium pays better than any other strategy profile and strictly better than any other "non-permutation" strategy profile when the prior satisfies some mild conditions.