# Convolutional Neural Networks For Automatic State-Time Feature Extraction in Reinforcement Learning Applied to Residential Load Control

arXiv: https://arxiv.org/abs/1604.08382 · HTML: https://mlsanity.com/paper/1604.08382/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2016.

## Built on by (most cited first, 6 of 6)

- [PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems](https://mlsanity.com/paper/2111.05969/) (2021)
- [DRL: Deep Reinforcement Learning for Intelligent Robot Control -- Concept, Literature, and Future](https://mlsanity.com/paper/2105.13806/) (2021)
- [Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks](https://mlsanity.com/paper/2202.05395/) (2022)
- [Computationally efficient joint coordination of multiple electric vehicle charging points using reinforcement learning](https://mlsanity.com/paper/2203.14078/) (2022)
- [Domain-adapted Learning and Imitation: DRL for Power Arbitrage](https://mlsanity.com/paper/2301.08360/) (2023)
- [Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning](https://mlsanity.com/paper/2406.07069/) (2024)

## Abstract (arXiv)

Direct load control of a heterogeneous cluster of residential demand flexibility sources is a high-dimensional control problem with partial observability. This work proposes a novel approach that uses a convolutional neural network to extract hidden state-time features to mitigate the curse of partial observability. More specific, a convolutional neural network is used as a function approximator to estimate the state-action value function or Q-function in the supervised learning step of fitted Q-iteration. The approach is evaluated in a qualitative simulation, comprising a cluster of thermostatically controlled loads that only share their air temperature, whilst their envelope temperature remains hidden. The simulation results show that the presented approach is able to capture the underlying hidden features and successfully reduce the electricity cost the cluster.