# Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Serving

arXiv: https://arxiv.org/abs/2504.17999 · HTML: https://mlsanity.com/paper/2504.17999/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving.
- Builds on 18 of its 91 references that are in the map.
- First posted on arXiv 25 Apr 2025.

## Placed in

- Efficient LLM inference and serving / [Serving systems and request scheduling](https://mlsanity.com/inference/serving-systems-and-request-scheduling/)

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks](https://mlsanity.com/paper/2303.03667/) (2023)
- [Andes: Defining and Enhancing Quality-of-Experience in LLM-Based Text Streaming Services](https://mlsanity.com/paper/2404.16283/) (2024)
- [GPTCoach: Towards LLM-Based Physical Activity Coaching](https://mlsanity.com/paper/2405.06061/) (2024)
- [Customizing Emotional Support: How Do Individuals Construct and Interact With LLM-Powered Chatbots](https://mlsanity.com/paper/2504.12943/) (2025)
- [Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scaling](https://mlsanity.com/paper/2501.08219/) (2025)
- [CLARE: Cognitive Load Assessment in REaltime with Multimodal Data](https://mlsanity.com/paper/2404.17098/) (2024)
- [Color-Perception-Guided Display Power Reduction for Virtual Reality](https://mlsanity.com/paper/2209.07610/) (2022)
- [A Survey and Measurement Study of GPU DVFS on Energy Conservation](https://mlsanity.com/paper/1610.01784/) (2016)
- [The Hidden Cost of Using Amazon Mechanical Turk for Research](https://mlsanity.com/paper/2101.04459/) (2021)
- [Predicting the Relative Difficulty of Single Sentences With and Without Surrounding Context](https://mlsanity.com/paper/1606.08425/) (2016)
- [ASHABot: An LLM-Powered Chatbot to Support the Informational Needs of Community Health Workers](https://mlsanity.com/paper/2409.10913/) (2024)
- [On the Contribution of Discourse Structure on Text Complexity Assessment](https://mlsanity.com/paper/1708.05800/) (2017)
- [Gaze-Driven Adaptive Interventions for Magazine-Style Narrative Visualizations](https://mlsanity.com/paper/1909.01379/) (2019)
- [Eloquent: A More Robust Transmission Scheme for LLM Token Streaming](https://mlsanity.com/paper/2401.12961/) (2024)
- [TimelyLLM: Segmented LLM Serving System for Time-sensitive Robotic Applications](https://mlsanity.com/paper/2412.18695/) (2024)

## Built on by (most cited first, 3 of 3)

- [TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling](https://mlsanity.com/paper/2510.02758/) (2025)
- [Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents](https://mlsanity.com/paper/2601.20412/) (2026)
- [SchemaWeaver: Accelerating On-Device Mobile Agentic AI via Schema-Guided Parallel LLM Decoding](https://mlsanity.com/paper/2610.17436/) (2026)

## Abstract (arXiv)

Generative conversational interfaces powered by large language models (LLMs) typically stream output token-by-token at a rate determined by computational budget, often neglecting actual human reading speeds and the cognitive load associated with the content. This mismatch frequently leads to inefficient use of computational resources. For example, in cloud-based services, streaming content faster than users can read appears unnecessary, resulting in wasted computational resources and potential delays for other users, particularly during peak usage periods. To address this issue, we propose an adaptive streaming method that dynamically adjusts the pacing of LLM streaming output in real-time based on inferred cognitive load. Our approach estimates the cognitive load associated with streaming content and strategically slows down the stream during complex or information-rich segments, thereby freeing computational resources for other users. We conducted a statistical analysis and simulation based on a statistical model derived from data collected in a crowdsourced user study across various types of LLM-generated content. Our results show that this adaptive method can effectively reduce computational consumption while largely maintaining streaming speed above user's normal reading speed.