# SA-Net: Deep Neural Network for Robot Trajectory Recognition from RGB-D Streams

arXiv: https://arxiv.org/abs/1905.04380 · HTML: https://mlsanity.com/paper/1905.04380/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 10 May 2019.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Built on by (most cited first, 1 of 1)

- [A Hierarchical Bayesian model for Inverse RL in Partially-Controlled Environments](https://mlsanity.com/paper/2107.05818/) (2021)

## Abstract (arXiv)

Learning from demonstration (LfD) and imitation learning offer new paradigms for transferring task behavior to robots. A class of methods that enable such online learning require the robot to observe the task being performed and decompose the sensed streaming data into sequences of state-action pairs, which are then input to the methods. Thus, recognizing the state-action pairs correctly and quickly in sensed data is a crucial prerequisite for these methods. We present SA-Net a deep neural network architecture that recognizes state-action pairs from RGB-D data streams. SA-Net performed well in two diverse robotic applications of LfD -- one involving mobile ground robots and another involving a robotic manipulator -- which demonstrates that the architecture generalizes well to differing contexts. Comprehensive evaluations including deployment on a physical robot show that sanet significantly improves on the accuracy of the previous method that utilizes traditional image processing and segmentation.