# Cognitive Manipulation: Semi-supervised Visual Representation and Classroom-to-real Reinforcement Learning for Assembly in Semi-structured Environments

arXiv: https://arxiv.org/abs/2406.00364 · HTML: https://mlsanity.com/paper/2406.00364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 51 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 14 of 14)

- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [Learning to See before Learning to Act: Visual Pre-training for Manipulation](https://mlsanity.com/paper/2107.00646/) (2021)
- [Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study](https://mlsanity.com/paper/2103.11512/) (2021)
- [Affordance Learning from Play for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2203.00352/) (2022)
- [Few-Shot Object Detection: A Comprehensive Survey](https://mlsanity.com/paper/2112.11699/) (2021)
- [Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2005.10872/) (2020)
- [A Framework for Robot Manipulation: Skill Formalism, Meta Learning and Adaptive Control](https://mlsanity.com/paper/1805.08576/) (2018)
- [Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty](https://mlsanity.com/paper/2106.04306/) (2021)
- [Fast robust peg-in-hole insertion with continuous visual servoing](https://mlsanity.com/paper/2011.06399/) (2020)
- [ManiPose: A Comprehensive Benchmark for Pose-aware Object Manipulation in Robotics](https://mlsanity.com/paper/2403.13365/) (2024)
- [Learning Skills from Demonstrations: A Trend from Motion Primitives to Experience Abstraction](https://mlsanity.com/paper/2210.08060/) (2022)
- [Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions](https://mlsanity.com/paper/2312.16438/) (2023)
- [Proactive Action Visual Residual Reinforcement Learning for Contact-Rich Tasks Using a Torque-Controlled Robot](https://mlsanity.com/paper/2010.13056/) (2020)
- [Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesis](https://mlsanity.com/paper/2301.01441/) (2023)

## Abstract (arXiv)

Assembling a slave object into a fixture-free master object represents a critical challenge in flexible manufacturing. Existing deep reinforcement learning-based methods, while benefiting from visual or operational priors, often struggle with small-batch precise assembly tasks due to their reliance on insufficient priors and high-costed model development. To address these limitations, this paper introduces a cognitive manipulation and learning approach that utilizes skill graphs to integrate learning-based object detection with fine manipulation models into a cohesive modular policy. This approach enables the detection of the master object from both global and local perspectives to accommodate positional uncertainties and variable backgrounds, and parametric residual policy to handle pose error and intricate contact dynamics effectively. Leveraging the skill graph, our method supports knowledge-informed learning of semi-supervised learning for object detection and classroom-to-real reinforcement learning for fine manipulation. Simulation experiments on a gear-assembly task have demonstrated that the skill-graph-enabled coarse-operation planning and visual attention are essential for efficient learning and robust manipulation, showing substantial improvements of 13% in success rate and 15.4% in number of completion steps over competing methods. Real-world experiments further validate that our system is highly effective for robotic assembly in semi-structured environments.