# Online Refinement of a Scene Recognition Model for Mobile Robots by Observing Human's Interaction with Environments

arXiv: https://arxiv.org/abs/2208.06636 · HTML: https://mlsanity.com/paper/2208.06636/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 13 Aug 2022.

## Builds on (most cited first, 11 of 11)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [Knowledge Distillation for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/1911.03462/) (2019)
- [Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels](https://mlsanity.com/paper/2005.06050/) (2020)
- [Object Finding in Cluttered Scenes Using Interactive Perception](https://mlsanity.com/paper/1911.07482/) (2019)
- [Image-based scene recognition for robot navigation considering traversable plants and its manual annotation-free training](https://mlsanity.com/paper/2108.00759/) (2021)

## Abstract (arXiv)

This paper describes a method of online refinement of a scene recognition model for robot navigation considering traversable plants, flexible plant parts which a robot can push aside while moving. In scene recognition systems that consider traversable plants growing out to the paths, misclassification may lead the robot to getting stuck due to the traversable plants recognized as obstacles. Yet, misclassification is inevitable in any estimation methods. In this work, we propose a framework that allows for refining a semantic segmentation model on the fly during the robot's operation. We introduce a few-shot segmentation based on weight imprinting for online model refinement without fine-tuning. Training data are collected via observation of a human's interaction with the plant parts. We propose novel robust weight imprinting to mitigate the effect of noise included in the masks generated by the interaction. The proposed method was evaluated through experiments using real-world data and shown to outperform an ordinary weight imprinting and provide competitive results to fine-tuning with model distillation while requiring less computational cost.