# Symbolic State Estimation with Predicates for Contact-Rich Manipulation Tasks

arXiv: https://arxiv.org/abs/2203.02468 · HTML: https://mlsanity.com/paper/2203.02468/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2022.

## Built on by (most cited first, 8 of 8)

- [Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models](https://mlsanity.com/paper/2401.03360/) (2023)
- [Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress](https://mlsanity.com/paper/2410.04640/) (2024)
- [DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability](https://mlsanity.com/paper/2306.13196/) (2023)
- [STAP: Sequencing Task-Agnostic Policies](https://mlsanity.com/paper/2210.12250/) (2022)
- [GenCHiP: Generating Robot Policy Code for High-Precision and Contact-Rich Manipulation Tasks](https://mlsanity.com/paper/2404.06645/) (2024)
- [MOSAIC: Skill-Centric Manipulation Planning with Physics Simulation](https://mlsanity.com/paper/2504.16738/) (2025)
- [Jointly Learning Predicates and Actions Enables Zero-Shot Skill Composition](https://mlsanity.com/paper/2605.20648/) (2026)
- [See and Switch: Vision-Based Branching for Interactive Robot-Skill Programming](https://mlsanity.com/paper/2603.08057/) (2026)

## Abstract (arXiv)

Manipulation tasks often require a robot to adjust its sensorimotor skills based on the state it finds itself in. Taking peg-in-hole as an example: once the peg is aligned with the hole, the robot should push the peg downwards. While high level execution frameworks such as state machines and behavior trees are commonly used to formalize such decision-making problems, these frameworks require a mechanism to detect the high-level symbolic state. Handcrafting heuristics to identify symbolic states can be brittle, and using data-driven methods can produce noisy predictions, particularly when working with limited datasets, as is common in real-world robotic scenarios. This paper proposes a Bayesian state estimation method to predict symbolic states with predicate classifiers. This method requires little training data and allows fusing noisy observations from multiple sensor modalities. We evaluate our framework on a set of real-world peg-in-hole and connector-socket insertion tasks, demonstrating its ability to classify symbolic states and to generalize to unseen tasks, outperforming baseline methods. We also demonstrate the ability of our method to improve the robustness of manipulation policies on a real robot.