# Learning to Guide Human Experts via Personalized Large Language Models

arXiv: https://arxiv.org/abs/2308.06039 · HTML: https://mlsanity.com/paper/2308.06039/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models](https://mlsanity.com/paper/2302.07257/) (2023)
- [The Algorithmic Automation Problem: Prediction, Triage, and Human Effort](https://mlsanity.com/paper/1903.12220/) (2019)
- [Differentiable Learning Under Triage](https://mlsanity.com/paper/2103.08902/) (2021)
- [Calibrated Learning to Defer with One-vs-All Classifiers](https://mlsanity.com/paper/2202.03673/) (2022)
- [Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making](https://mlsanity.com/paper/2010.07938/) (2020)
- [Regression Under Human Assistance](https://mlsanity.com/paper/1909.02963/) (2019)
- [RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reporting](https://mlsanity.com/paper/2107.02104/) (2021)
- [Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosis](https://mlsanity.com/paper/2108.07392/) (2021)
- [Designing Closed Human-in-the-loop Deferral Pipelines](https://mlsanity.com/paper/2202.04718/) (2022)
- [Reinforcing an Image Caption Generator Using Off-Line Human Feedback](https://mlsanity.com/paper/1911.09753/) (2019)

## Abstract (arXiv)

In learning to defer, a predictor identifies risky decisions and defers them to a human expert. One key issue with this setup is that the expert may end up over-relying on the machine's decisions, due to anchoring bias. At the same time, whenever the machine chooses the deferral option the expert has to take decisions entirely unassisted. As a remedy, we propose learning to guide (LTG), an alternative framework in which -- rather than suggesting ready-made decisions -- the machine provides guidance useful to guide decision-making, and the human is entirely responsible for coming up with a decision. We also introduce SLOG, an LTG implementation that leverages (a small amount of) human supervision to convert a generic large language model into a module capable of generating textual guidance, and present preliminary but promising results on a medical diagnosis task.