# Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp Diffusion

arXiv: https://arxiv.org/abs/2609.33546 · HTML: https://mlsanity.com/paper/2609.33546/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 24 of its 25 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 24 of 24)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [Practical and Asymptotically Exact Conditional Sampling in Diffusion Models](https://mlsanity.com/paper/2306.17775/) (2023)
- [A General Framework for Inference-time Scaling and Steering of Diffusion Models](https://mlsanity.com/paper/2501.06848/) (2025)
- [SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion](https://mlsanity.com/paper/2209.03855/) (2022)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Diffusion Predictive Control with Constraints](https://mlsanity.com/paper/2412.09342/) (2024)
- [GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training](https://mlsanity.com/paper/2507.13097/) (2025)
- [Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design](https://mlsanity.com/paper/2502.14944/) (2025)
- [Robotic Grasping from Classical to Modern: A Survey](https://mlsanity.com/paper/2202.03631/) (2022)
- [DA2 Dataset: Toward Dexterity-Aware Dual-Arm Grasping](https://mlsanity.com/paper/2208.00408/) (2022)
- [cuRoboV2: Dynamics-Aware Motion Generation with Depth-Fused Distance Fields for High-DoF Robots](https://mlsanity.com/paper/2603.05493/) (2026)
- [Learning to Generate 6-DoF Grasp Poses with Reachability Awareness](https://mlsanity.com/paper/1910.06404/) (2019)
- [Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion](https://mlsanity.com/paper/2605.23346/) (2026)
- [GraspGen-X: Cross-Embodiment 6-DOF Diffusion-based Grasping](https://mlsanity.com/paper/2606.00998/) (2026)
- [EmbodiSteer: Steering Embodiment-Agnostic Visuomotor Policies with Joint-Space Guidance for Zero-Shot Cross-Embodiment Deployment](https://mlsanity.com/paper/2606.12965/) (2026)
- [Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control](https://mlsanity.com/paper/2606.24208/) (2026)
- [DG16M: A Large-Scale Dataset for Dual-Arm Grasping with Force-Optimized Grasps](https://mlsanity.com/paper/2503.08358/) (2025)
- [BiGraspFormer: End-to-End Bimanual Grasp Transformer](https://mlsanity.com/paper/2509.19142/) (2025)
- [DAGDiff: Guiding Dual-Arm Grasp Diffusion to Stable and Collision-Free Grasps](https://mlsanity.com/paper/2509.21145/) (2025)
- [Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models](https://mlsanity.com/paper/2608.16351/) (2026)

## Abstract (arXiv)

Current grasp diffusion models provide rich priors for generation, yet their object-centric approach can violate the kinematic and collision constraints imposed by the embodiment and the environment. Existing embodiment-aware methods primarily perform local corrections around generated grasps through gradient guidance or optimization, making it difficult to recover from fundamentally infeasible modes. We present Steer2Grasp, a training-free, embodiment-agnostic framework for inference-time grasp steering that adapts a frozen Cartesian grasp diffusion model using deployment-specific rewards. Through Feynman-Kac (FK) inspired particle reweighting and resampling, the method reallocates population mass from infeasible to high-reward grasp modes, enabling population-level mode transitions without modifying the pretrained diffusion model or requiring differentiable constraints. The framework enables a unified treatment for single and dual arm grasping through reachability and collision aware rewards, followed by gradient free gripper level local refinement. Across diverse objects, robot embodiments, and constrained environments, our method substantially improves feasible grasp generation while maintaining proximity to the underlying grasp prior.