# PhysicsAssistant: An LLM-Powered Interactive Learning Robot for Physics Lab Investigations

arXiv: https://arxiv.org/abs/2403.18721 · HTML: https://mlsanity.com/paper/2403.18721/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2024.

## Built on by (most cited first, 8 of 8)

- [LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs](https://mlsanity.com/paper/2410.14182/) (2024)
- [Safety Aware Task Planning via Large Language Models in Robotics](https://mlsanity.com/paper/2503.15707/) (2025)
- [Transforming Teachers' Roles and Agencies in the Era of Generative AI: Perceptions, Acceptance, Knowledge, and Practices](https://mlsanity.com/paper/2410.03018/) (2024)
- [Realizing Visual Question Answering for Education: GPT-4V as a Multimodal AI](https://mlsanity.com/paper/2405.07163/) (2024)
- [Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories](https://mlsanity.com/paper/2501.06143/) (2025)
- [PhysLab: A Benchmark Dataset for Multi-Granularity Visual Parsing of Physics Experiments](https://mlsanity.com/paper/2506.06631/) (2025)
- [Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplers](https://mlsanity.com/paper/2602.18292/) (2026)
- [See-Control: A Multimodal Agent Framework for Smartphone Interaction with a Robotic Arm](https://mlsanity.com/paper/2512.08629/) (2025)

## Abstract (arXiv)

Robot systems in education can leverage Large language models' (LLMs) natural language understanding capabilities to provide assistance and facilitate learning. This paper proposes a multimodal interactive robot (PhysicsAssistant) built on YOLOv8 object detection, cameras, speech recognition, and chatbot using LLM to provide assistance to students' physics labs. We conduct a user study on ten 8th-grade students to empirically evaluate the performance of PhysicsAssistant with a human expert. The Expert rates the assistants' responses to student queries on a 0-4 scale based on Bloom's taxonomy to provide educational support. We have compared the performance of PhysicsAssistant (YOLOv8+GPT-3.5-turbo) with GPT-4 and found that the human expert rating of both systems for factual understanding is the same. However, the rating of GPT-4 for conceptual and procedural knowledge (3 and 3.2 vs 2.2 and 2.6, respectively) is significantly higher than PhysicsAssistant (p < 0.05). However, the response time of GPT-4 is significantly higher than PhysicsAssistant (3.54 vs 1.64 sec, p < 0.05). Hence, despite the relatively lower response quality of PhysicsAssistant than GPT-4, it has shown potential for being used as a real-time lab assistant to provide timely responses and can offload teachers' labor to assist with repetitive tasks. To the best of our knowledge, this is the first attempt to build such an interactive multimodal robotic assistant for K-12 science (physics) education.