# OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models

arXiv: https://arxiv.org/abs/2508.21061 · HTML: https://mlsanity.com/paper/2508.21061/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [LLM Critics Help Catch LLM Bugs](https://mlsanity.com/paper/2407.00215/) (2024)
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://mlsanity.com/paper/2203.06566/) (2022)
- [Survey on Evaluation Methods for Dialogue Systems](https://mlsanity.com/paper/1905.04071/) (2019)
- [The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development](https://mlsanity.com/paper/2302.07080/) (2023)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- [Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces](https://mlsanity.com/paper/2309.14459/) (2023)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models](https://mlsanity.com/paper/2305.11483/) (2023)
- [A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration](https://mlsanity.com/paper/2404.00405/) (2024)
- [Beyond the Chat: Executable and Verifiable Text-Editing with LLMs](https://mlsanity.com/paper/2309.15337/) (2023)
- [DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models](https://mlsanity.com/paper/2310.03691/) (2023)
- [WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback](https://mlsanity.com/paper/2408.15549/) (2024)
- [Supporting Sensemaking of Large Language Model Outputs at Scale](https://mlsanity.com/paper/2401.13726/) (2024)
- [Understanding Users' Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Level](https://mlsanity.com/paper/2311.07434/) (2023)
- [Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation](https://mlsanity.com/paper/2405.13037/) (2024)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models](https://mlsanity.com/paper/2304.01964/) (2023)
- [LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History](https://mlsanity.com/paper/2402.18216/) (2024)
- [SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking](https://mlsanity.com/paper/2402.02285/) (2024)
- [Task Supportive and Personalized Human-Large Language Model Interaction: A User Study](https://mlsanity.com/paper/2402.06170/) (2024)
- [Conversational AI Threads for Visualizing Multidimensional Datasets](https://mlsanity.com/paper/2311.05590/) (2023)
- [How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots](https://mlsanity.com/paper/2404.17025/) (2024)
- [StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions](https://mlsanity.com/paper/2407.12423/) (2024)
- [Developing a Conversational Recommendation System for Navigating Limited Options](https://mlsanity.com/paper/2104.06552/) (2021)

## Built on by (most cited first, 7 of 7)

- [When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks](https://mlsanity.com/paper/2510.05307/) (2025)
- [Alignment-Process-Outcome: Rethinking How AIs and Humans Collaborate](https://mlsanity.com/paper/2603.08017/) (2026)
- [OrchVis: Hierarchical Multi-Agent Orchestration for Human Oversight](https://mlsanity.com/paper/2510.24937/) (2025)
- [Mixed-Initiative Context: Structuring and Managing Context for Human-AI Collaboration](https://mlsanity.com/paper/2604.07121/) (2026)
- [CogInstrument: Modeling Cognitive Processes for Bidirectional Human-LLM Alignment in Planning Tasks](https://mlsanity.com/paper/2604.10587/) (2026)
- [Choose Your Own Adventure: Non-Linear AI-Assisted Programming with EvoGraph](https://mlsanity.com/paper/2604.18883/) (2026)
- ["What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Use](https://mlsanity.com/paper/2605.00497/) (2026)

## Abstract (arXiv)

As multi-turn dialogues with large language models (LLMs) grow longer and more complex, how can users better evaluate and review progress on their conversational goals? We present OnGoal, an LLM chat interface that helps users better manage goal progress. OnGoal provides real-time feedback on goal alignment through LLM-assisted evaluation, explanations for evaluation results with examples, and overviews of goal progression over time, enabling users to navigate complex dialogues more effectively. Through a study with 20 participants on a writing task, we evaluate OnGoal against a baseline chat interface without goal tracking. Using OnGoal, participants spent less time and effort to achieve their goals while exploring new prompting strategies to overcome miscommunication, suggesting tracking and visualizing goals can enhance engagement and resilience in LLM dialogues. Our findings inspired design implications for future LLM chat interfaces that improve goal communication, reduce cognitive load, enhance interactivity, and enable feedback to improve LLM performance.