# Semantic Prompting: Agentic Incremental Narrative Refinement through Spatial Semantic Interaction

arXiv: https://arxiv.org/abs/2604.19971 · HTML: https://mlsanity.com/paper/2604.19971/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 23 of 23)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model](https://mlsanity.com/paper/2010.09142/) (2020)
- [The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers](https://mlsanity.com/paper/2101.09157/) (2021)
- [Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries](https://mlsanity.com/paper/2208.09323/) (2022)
- [Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision](https://mlsanity.com/paper/2204.03685/) (2022)
- [VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping](https://mlsanity.com/paper/2304.07810/) (2023)
- [The Value, Benefits, and Concerns of Generative AI-Powered Assistance in Writing](https://mlsanity.com/paper/2403.12004/) (2024)
- [Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting](https://mlsanity.com/paper/2303.03199/) (2023)
- [LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia](https://mlsanity.com/paper/2207.02308/) (2022)
- [Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools](https://mlsanity.com/paper/2405.17217/) (2024)
- [Script&Shift: A Layered Interface Paradigm for Integrating Content Development and Rhetorical Strategy with LLM Writing Assistants](https://mlsanity.com/paper/2502.10638/) (2025)
- [SARD: A Human-AI Collaborative Story Generation](https://mlsanity.com/paper/2403.01575/) (2024)
- [DeepSI: Interactive Deep Learning for Semantic Interaction](https://mlsanity.com/paper/2305.18357/) (2023)
- [Compositional Structures as Substrates for Human-AI Co-creation Environment: A Design Approach and A Case Study](https://mlsanity.com/paper/2503.04103/) (2025)
- [Steering LLM Summarization with Visual Workspaces for Sensemaking](https://mlsanity.com/paper/2409.17289/) (2024)
- [DATAWEAVER: Authoring Data-Driven Narratives through the Integrated Composition of Visualization and Text](https://mlsanity.com/paper/2503.22946/) (2025)
- [LLM-driven Constrained Copy Generation through Iterative Refinement](https://mlsanity.com/paper/2504.10391/) (2025)
- [NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modification](https://mlsanity.com/paper/2508.02823/) (2025)

## Abstract (arXiv)

Interactive spatial layouts empower users to synthesize information and organize findings for sensemaking. While Large Language Models (LLMs) can automate narrative generation from spatial layouts, current collage-based and re-generation methods struggle to support the incremental spatial refinements inherent to the sensemaking process. We identify three critical gaps in existing spatial-textual generation: interaction-revision misalignment, human-LLM intent misalignment, and lack of granular customization. To address these, we introduce Semantic Prompting, a framework for spatial refinement that perceives semantic interactions, reasons about refinement intent, and performs targeted positional revisions. We implemented S-PRISM to realize this framework. The empirical evaluation demonstrated that S-PRISM effectively enhanced the precision of interaction-revision refinement. A user study (N=14) highlighted how participants leveraged S-PRISM for incremental formalization through interactive steering. Results showed that users valued its efficient, adaptable, and trustworthy support, which effectively strengthens human-LLM intent alignment.