# Harmony: A Human-Aware, Responsive, Modular Assistant with a Locally Deployed Large Language Model

arXiv: https://arxiv.org/abs/2410.14252 · HTML: https://mlsanity.com/paper/2410.14252/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2024.

## Built on by (most cited first, 2 of 2)

- [SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes](https://mlsanity.com/paper/2606.01912/) (2026)
- [AdaHome: An Adaptive Smart Home Assistant using Local Small Language Models](https://mlsanity.com/paper/2607.18034/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) offer powerful capabilities for natural language understanding, enabling more intelligent smart home assistants. However, existing systems often rely on cloud-based LLMs, raising concerns around user privacy and system dependency on external connectivity. In this work, we present Harmony, a privacy-preserving and robust smart home assistant powered by the locally deployable Llama3-8B model. Beyond protecting user data, Harmony also addresses reliability challenges of smaller models, such as hallucination and instruction misinterpretation, through structured prompting and modular agent design. Experimental results in both virtual environments and user studies show that Harmony achieves performance comparable to GPT-4-based systems, while enabling offline, proactive, and personalized smart home interaction.