# Potential Impacts of Smart Homes on Human Behavior: A Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2102.13307 · HTML: https://mlsanity.com/paper/2102.13307/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 51 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Trial without Error: Towards Safe Reinforcement Learning via Human Intervention](https://mlsanity.com/paper/1707.05173/) (2017)
- [Dialogue Learning With Human-In-The-Loop](https://mlsanity.com/paper/1611.09823/) (2016)
- [Leveraging Human Guidance for Deep Reinforcement Learning Tasks](https://mlsanity.com/paper/1909.09906/) (2019)
- [Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari](https://mlsanity.com/paper/1802.08842/) (2018)
- [Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning](https://mlsanity.com/paper/1901.04693/) (2019)
- [Human-Like Decision Making: Document-level Aspect Sentiment Classification via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1910.09260/) (2019)
- [Hierarchical Reinforcement Learning as a Model of Human Task Interleaving](https://mlsanity.com/paper/2001.02122/) (2020)

## Built on by (most cited first, 1 of 1)

- [Towards Personalization of User Preferences in Partially Observable Smart Home Environments](https://mlsanity.com/paper/2112.00971/) (2021)

## Abstract (arXiv)

We aim to investigate the potential impacts of smart homes on human behavior. To this end, we simulate a series of human models capable of performing various activities inside a reinforcement learning-based smart home. We then investigate the possibility of human behavior being altered as a result of the smart home and the human model adapting to one-another. We design a semi-Markov decision process human task interleaving model based on hierarchical reinforcement learning that learns to make decisions to either pursue or leave an activity. We then integrate our human model in the smart home which is based on Q-learning. We show that a smart home trained on a generic human model is able to anticipate and learn the thermal preferences of human models with intrinsic rewards similar to the generic model. The hierarchical human model learns to complete each activity and set optimal thermal settings for maximum comfort. With the smart home, the number of time steps required to change the thermal settings are reduced for the human models. Interestingly, we observe that small variations in the human model reward structures can lead to the opposite behavior in the form of unexpected switching between activities which signals changes in human behavior due to the presence of the smart home.