# Supporting Sensemaking of Large Language Model Outputs at Scale

arXiv: https://arxiv.org/abs/2401.13726 · HTML: https://mlsanity.com/paper/2401.13726/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 67 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 6 of 6)

- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers](https://mlsanity.com/paper/2101.09157/) (2021)
- [LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia](https://mlsanity.com/paper/2207.02308/) (2022)

## Built on by (most cited first, 27 of 27)

- [Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences](https://mlsanity.com/paper/2404.12272/) (2024)
- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- [AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Tools](https://mlsanity.com/paper/2502.18736/) (2025)
- [OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models](https://mlsanity.com/paper/2508.21061/) (2025)
- [Exploring Empty Spaces: Human-in-the-Loop Data Augmentation](https://mlsanity.com/paper/2410.01088/) (2024)
- [Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions](https://mlsanity.com/paper/2507.15692/) (2025)
- [CorpusStudio: Surfacing Emergent Patterns in a Corpus of Prior Work while Writing](https://mlsanity.com/paper/2503.12436/) (2025)
- [From Words to Widgets for Controllable LLM Generation](https://mlsanity.com/paper/2604.10925/) (2026)
- [Large Language Models Meet User Interfaces: The Case of Provisioning Feedback](https://mlsanity.com/paper/2404.11072/) (2024)
- [What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns](https://mlsanity.com/paper/2504.15815/) (2025)
- [Bridging Dictionary: AI-Generated Dictionary of Partisan Language Use](https://mlsanity.com/paper/2407.09661/) (2024)
- [DxHF: Providing High-Quality Human Feedback for LLM Alignment via Interactive Decomposition](https://mlsanity.com/paper/2507.18802/) (2025)
- [Evalet: Evaluating Large Language Models through Functional Fragmentation](https://mlsanity.com/paper/2509.11206/) (2025)
- [Improving User Interface Generation Models from Designer Feedback](https://mlsanity.com/paper/2509.16779/) (2025)
- [Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers](https://mlsanity.com/paper/2510.00361/) (2025)
- [ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Models](https://mlsanity.com/paper/2511.11187/) (2025)
- [Understanding User Experiences of Computer Use Agents: Design Space and Opportunities for Building Agent UX Prototypes](https://mlsanity.com/paper/2510.04452/) (2025)
- [Texterial: A Text-as-Material Interaction Paradigm for LLM-Mediated Writing](https://mlsanity.com/paper/2603.00452/) (2026)
- [HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding](https://mlsanity.com/paper/2505.22906/) (2025)
- [Understanding Prompt Programming Tasks and Questions](https://mlsanity.com/paper/2507.17264/) (2025)
- [Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wild](https://mlsanity.com/paper/2605.05767/) (2026)
- [A Human-Centered Approach to Identifying Promises, Risks, & Challenges of Text-to-Image Generative AI in Radiology](https://mlsanity.com/paper/2507.16207/) (2025)
- [InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation](https://mlsanity.com/paper/2512.10234/) (2025)
- [Mixed-Initiative Context: Structuring and Managing Context for Human-AI Collaboration](https://mlsanity.com/paper/2604.07121/) (2026)
- [Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations](https://mlsanity.com/paper/2604.18724/) (2026)
- [Visual Fingerprints for LLM Generation Comparison](https://mlsanity.com/paper/2605.06054/) (2026)
- [DimSteer: Steering LLM Authoring with Automatically Discovered Stylistic Controls](https://mlsanity.com/paper/2610.04174/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are capable of generating multiple responses to a single prompt, yet little effort has been expended to help end-users or system designers make use of this capability. In this paper, we explore how to present many LLM responses at once. We design five features, which include both pre-existing and novel methods for computing similarities and differences across textual documents, as well as how to render their outputs. We report on a controlled user study (n=24) and eight case studies evaluating these features and how they support users in different tasks. We find that the features support a wide variety of sensemaking tasks and even make tasks previously considered to be too difficult by our participants now tractable. Finally, we present design guidelines to inform future explorations of new LLM interfaces.