# Thoughtful Things: Building Human-Centric Smart Devices with Small Language Models

arXiv: https://arxiv.org/abs/2405.03821 · HTML: https://mlsanity.com/paper/2405.03821/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 64 references that are in the map.
- First posted on arXiv 6 May 2024.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- ["Get ready for a party": Exploring smarter smart spaces with help from large language models](https://mlsanity.com/paper/2303.14143/) (2023)
- [SmartEx: A Framework for Generating User-Centric Explanations in Smart Environments](https://mlsanity.com/paper/2402.13024/) (2024)
- [Smart Home Appliances: Chat with Your Fridge](https://mlsanity.com/paper/1912.09589/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Survey on Collaborative Mechanisms Between Large and Small Language Models](https://mlsanity.com/paper/2505.07460/) (2025)
- [AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environments](https://mlsanity.com/paper/2506.11773/) (2025)
- [Creation of AI-driven Smart Spaces for Enhanced Indoor Environments -- A Survey](https://mlsanity.com/paper/2412.14708/) (2024)
- [Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models](https://mlsanity.com/paper/2507.08878/) (2025)

## Abstract (arXiv)

Everyday devices like light bulbs and kitchen appliances are now embedded with so many features and automated behaviors that they have become complicated to actually use. While such "smart" capabilities can better support users' goals, the task of learning the "ins and outs" of different devices is daunting. Voice assistants aim to solve this problem by providing a natural language interface to devices, yet such assistants cannot understand loosely-constrained commands, they lack the ability to reason about and explain devices' behaviors to users, and they rely on connectivity to intrusive cloud infrastructure. Toward addressing these issues, we propose thoughtful things: devices that leverage lightweight, on-device language models to take actions and explain their behaviors in response to unconstrained user commands. We propose an end-to-end framework that leverages formal modeling, automated training data synthesis, and generative language models to create devices that are both capable and thoughtful in the presence of unconstrained user goals and inquiries. Our framework requires no labeled data and can be deployed on-device, with no cloud dependency. We implement two thoughtful things (a lamp and a thermostat) and deploy them on real hardware, evaluating their practical performance.