# CueTip: An Interactive and Explainable Physics-aware Pool Assistant

arXiv: https://arxiv.org/abs/2501.18291 · HTML: https://mlsanity.com/paper/2501.18291/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 98 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 27 of 27)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [On the Planning Abilities of Large Language Models : A Critical Investigation](https://mlsanity.com/paper/2305.15771/) (2023)
- [LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks](https://mlsanity.com/paper/2402.01817/) (2024)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [Mind's Eye: Grounded Language Model Reasoning through Simulation](https://mlsanity.com/paper/2210.05359/) (2022)
- [A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision](https://mlsanity.com/paper/2308.07898/) (2023)
- [Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles](https://mlsanity.com/paper/2407.00870/) (2024)
- [Measuring the Mixing of Contextual Information in the Transformer](https://mlsanity.com/paper/2203.04212/) (2022)
- [Learning Visual Predictive Models of Physics for Playing Billiards](https://mlsanity.com/paper/1511.07404/) (2015)
- [LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Engines](https://mlsanity.com/paper/2411.08027/) (2024)
- [GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language models](https://mlsanity.com/paper/2311.09048/) (2023)
- [Using Large Language Model to Solve and Explain Physics Word Problems Approaching Human Level](https://mlsanity.com/paper/2309.08182/) (2023)
- [A little less conversation, a little more action, please: Investigating the physical common-sense of LLMs in a 3D embodied environment](https://mlsanity.com/paper/2410.23242/) (2024)
- [A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course](https://mlsanity.com/paper/2403.16977/) (2024)
- [Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation](https://mlsanity.com/paper/2411.00412/) (2024)
- [SimLM: Can Language Models Infer Parameters of Physical Systems?](https://mlsanity.com/paper/2312.14215/) (2023)

## Built on by (most cited first, 3 of 3)

- [Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI](https://mlsanity.com/paper/2510.04978/) (2025)
- [Discovering High Level Patterns from Simulation Traces](https://mlsanity.com/paper/2602.10009/) (2026)
- [Text2Sim: Agentic Physics-Based Simulation Generation with Distilled Expertise](https://mlsanity.com/paper/2609.36593/) (2026)

## Abstract (arXiv)

We present an interactive and explainable automated coaching assistant called CueTip for a variant of pool/billiards. CueTip's novelty lies in its combination of three features: a natural-language interface, an ability to perform contextual, physics-aware reasoning, and that its explanations are rooted in a set of predetermined guidelines developed by domain experts. We instrument a physics simulator so that it generates event traces in natural language alongside traditional state traces. Event traces lend themselves to interpretation by language models, which serve as the interface to our assistant. We design and train a neural adaptor that decouples tactical choices made by CueTip from its interactivity and explainability allowing it to be reconfigured to mimic any pool playing agent. Our experiments show that CueTip enables contextual query-based assistance and explanations while maintaining the strength of the agent in terms of win rate (improving it in some situations). The explanations generated by CueTip are physically-aware and grounded in the expert rules and are therefore more reliable.