# Don't Forget to Buy Milk: Contextually Aware Grocery Reminder Household Robot

arXiv: https://arxiv.org/abs/2207.09050 · HTML: https://mlsanity.com/paper/2207.09050/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 35 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 7 of 7)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Learning Novel Objects Continually Through Curiosity](https://mlsanity.com/paper/2103.07758/) (2021)
- [A Large-Scale Deep Architecture for Personalized Grocery Basket Recommendations](https://mlsanity.com/paper/1910.12757/) (2019)

## Built on by (most cited first, 2 of 2)

- [Few-Shot Continual Active Learning by a Robot](https://mlsanity.com/paper/2210.04137/) (2022)
- [A Personalized Household Assistive Robot that Learns and Creates New Breakfast Options through Human-Robot Interaction](https://mlsanity.com/paper/2307.00114/) (2023)

## Abstract (arXiv)

Assistive robots operating in household environments would require items to be available in the house to perform assistive tasks. However, when these items run out, the assistive robot must remind its user to buy the missing items. In this paper, we present a computational architecture that can allow a robot to learn personalized contextual knowledge of a household through interactions with its user. The architecture can then use the learned knowledge to make predictions about missing items from the household over a long period of time. The architecture integrates state-of-the-art perceptual learning algorithms, cognitive models of memory encoding and learning, a reasoning module for predicting missing items from the household, and a graphical user interface (GUI) to interact with the user. The architecture is integrated with the Fetch mobile manipulator robot and validated in a large indoor environment with multiple contexts and objects. Our experimental results show that the robot can adapt to an environment by learning contextual knowledge through interactions with its user. The robot can also use the learned knowledge to correctly predict missing items over multiple weeks and it is robust against sensory and perceptual errors.