# Autonomous Curiosity for Real-Time Training Onboard Robotic Agents

arXiv: https://arxiv.org/abs/2109.00927 · HTML: https://mlsanity.com/paper/2109.00927/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 29 Aug 2021.

## Builds on (most cited first, 27 of 27)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [End-to-end Learning of Action Detection from Frame Glimpses in Videos](https://mlsanity.com/paper/1511.06984/) (2015)
- [Playing FPS Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05521/) (2016)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [End-to-end representation learning for Correlation Filter based tracking](https://mlsanity.com/paper/1704.06036/) (2017)
- [Learning how to Active Learn: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1708.02383/) (2017)
- [Reinforcement Learning for UAV Attitude Control](https://mlsanity.com/paper/1804.04154/) (2018)
- [Active Object Localization with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06015/) (2015)
- [Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks](https://mlsanity.com/paper/1709.00507/) (2017)
- [Active Learning for Convolutional Neural Networks: A Core-Set Approach](https://mlsanity.com/paper/1708.00489/) (2017)
- [End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning](https://mlsanity.com/paper/1808.03405/) (2018)
- [We don't need no bounding-boxes: Training object class detectors using only human verification](https://mlsanity.com/paper/1602.08405/) (2016)
- [Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motion](https://mlsanity.com/paper/1605.00164/) (2016)
- [Spot On: Action Localization from Pointly-Supervised Proposals](https://mlsanity.com/paper/1604.07602/) (2016)
- [Click Carving: Segmenting Objects in Video with Point Clicks](https://mlsanity.com/paper/1607.01115/) (2016)
- [On Learning Where To Look](https://mlsanity.com/paper/1405.5488/) (2014)

## Abstract (arXiv)

Learning requires both study and curiosity. A good learner is not only good at extracting information from the data given to it, but also skilled at finding the right new information to learn from. This is especially true when a human operator is required to provide the ground truth - such a source should only be queried sparingly. In this work, we address the problem of curiosity as it relates to online, real-time, human-in-the-loop training of an object detection algorithm onboard a robotic platform, one where motion produces new views of the subject. We propose a deep reinforcement learning approach that decides when to ask the human user for ground truth, and when to move. Through a series of experiments, we demonstrate that our agent learns a movement and request policy that is at least 3x more effective at using human user interactions to train an object detector than untrained approaches, and is generalizable to a variety of subjects and environments.