# Sensor Control for Information Gain in Dynamic, Sparse and Partially Observed Environments

arXiv: https://arxiv.org/abs/2211.01527 · HTML: https://mlsanity.com/paper/2211.01527/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning](https://mlsanity.com/paper/1803.00101/) (2018)
- [Model-based Reinforcement Learning: A Survey](https://mlsanity.com/paper/2006.16712/) (2020)
- [Recurrent Attention Models for Depth-Based Person Identification](https://mlsanity.com/paper/1611.07212/) (2016)
- [Learning in POMDPs with Monte Carlo Tree Search](https://mlsanity.com/paper/1806.05631/) (2018)
- [End-to-end Learning from Spectrum Data: A Deep Learning approach for Wireless Signal Identification in Spectrum Monitoring applications](https://mlsanity.com/paper/1712.03987/) (2017)
- [Maximizing Information Gain in Partially Observable Environments via Prediction Reward](https://mlsanity.com/paper/2005.04912/) (2020)
- [Exploiting Submodular Value Functions For Scaling Up Active Perception](https://mlsanity.com/paper/2009.09696/) (2020)
- [GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback](https://mlsanity.com/paper/2104.06600/) (2021)
- [Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing](https://mlsanity.com/paper/1804.07047/) (2018)
- [Spectral Attention-Driven Intelligent Target Signal Identification on a Wideband Spectrum](https://mlsanity.com/paper/1901.11368/) (2019)

## Abstract (arXiv)

We present an approach for autonomous sensor control for information gathering under partially observable, dynamic and sparsely sampled environments that maximizes information about entities present in that space. We describe our approach for the task of Radio-Frequency (RF) spectrum monitoring, where the goal is to search for and track unknown, dynamic signals in the environment. To this end, we extend the Deep Anticipatory Network (DAN) Reinforcement Learning (RL) framework by (1) improving exploration in sparse, non-stationary environments using a novel information gain reward, and (2) scaling up the control space and enabling the monitoring of complex, dynamic activity patterns using hybrid convolutional-recurrent neural layers. We also extend this problem to situations in which sampling from the intended RF spectrum/field is limited and propose a model-based version of the original RL algorithm that fine-tunes the controller via a model that is iteratively improved from the limited field sampling. Results in simulated RF environments of differing complexity show that our system outperforms the standard DAN architecture and is more flexible and robust than baseline expert-designed agents. We also show that it is adaptable to non-stationary emission environments.