# EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulation

arXiv: https://arxiv.org/abs/2310.16050 · HTML: https://mlsanity.com/paper/2310.16050/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 39 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 30 of 35)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Scaling Robot Learning with Semantically Imagined Experience](https://mlsanity.com/paper/2302.11550/) (2023)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [Generalization in Reinforcement Learning by Soft Data Augmentation](https://mlsanity.com/paper/2011.13389/) (2020)
- [Equivariant Point Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2103.14147/) (2021)
- [FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding](https://mlsanity.com/paper/2105.03655/) (2021)
- [Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds](https://mlsanity.com/paper/2106.06020/) (2021)
- [Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning](https://mlsanity.com/paper/2206.08321/) (2022)
- [FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy](https://mlsanity.com/paper/2111.05623/) (2021)
- [SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields](https://mlsanity.com/paper/2211.09786/) (2022)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Neural Grasp Distance Fields for Robot Manipulation](https://mlsanity.com/paper/2211.02647/) (2022)
- [VN-Transformer: Rotation-Equivariant Attention for Vector Neurons](https://mlsanity.com/paper/2206.04176/) (2022)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance](https://mlsanity.com/paper/2302.14268/) (2023)
- [ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation](https://mlsanity.com/paper/2203.06856/) (2022)
- [SEIL: Simulation-augmented Equivariant Imitation Learning](https://mlsanity.com/paper/2211.00194/) (2022)
- [Homogeneous vector bundles and G-equivariant convolutional neural networks](https://mlsanity.com/paper/2105.05400/) (2021)
- [Rotationally Equivariant 3D Object Detection](https://mlsanity.com/paper/2204.13630/) (2022)

## Built on by (most cited first, 30 of 37)

- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning](https://mlsanity.com/paper/2407.01479/) (2024)
- [Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress](https://mlsanity.com/paper/2410.04640/) (2024)
- [Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning](https://mlsanity.com/paper/2501.06994/) (2025)
- [Estimating Body and Hand Motion in an Ego-sensed World](https://mlsanity.com/paper/2410.03665/) (2024)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [ActionFlow: Equivariant, Accurate, and Efficient Policies with Spatially Symmetric Flow Matching](https://mlsanity.com/paper/2409.04576/) (2024)
- [ET-SEED: Efficient Trajectory-Level SE(3) Equivariant Diffusion Policy](https://mlsanity.com/paper/2411.03990/) (2024)
- [A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation](https://mlsanity.com/paper/2405.17418/) (2024)
- [VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation](https://mlsanity.com/paper/2407.04152/) (2024)
- [Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies](https://mlsanity.com/paper/2406.11740/) (2024)
- [A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI](https://mlsanity.com/paper/2505.01458/) (2025)
- [Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environments](https://mlsanity.com/paper/2312.09138/) (2023)
- [Leveraging Locality to Boost Sample Efficiency in Robotic Manipulation](https://mlsanity.com/paper/2406.10615/) (2024)
- [RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentation](https://mlsanity.com/paper/2403.19460/) (2024)
- [SE(3)-Equivariant Robot Learning and Control: A Tutorial Survey](https://mlsanity.com/paper/2503.09829/) (2025)
- [Mobi-π: Mobilizing Your Robot Learning Policy](https://mlsanity.com/paper/2505.23692/) (2025)
- [SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space](https://mlsanity.com/paper/2507.01723/) (2025)
- [EqvAfford: SE(3) Equivariance for Point-Level Affordance Learning](https://mlsanity.com/paper/2408.01953/) (2024)
- [MATCH POLICY: A Simple Pipeline from Point Cloud Registration to Manipulation Policies](https://mlsanity.com/paper/2409.15517/) (2024)
- [Constraint-Preserving Data Generation for Visuomotor Policy Learning](https://mlsanity.com/paper/2508.03944/) (2025)
- [A Practical Guide for Incorporating Symmetry in Diffusion Policy](https://mlsanity.com/paper/2505.13431/) (2025)
- [SE(3)-bi-equivariant Transformers for Point Cloud Assembly](https://mlsanity.com/paper/2407.09167/) (2024)
- [Non-rigid Relative Placement through 3D Dense Diffusion](https://mlsanity.com/paper/2410.19247/) (2024)
- [Learning Generalizable Shape Completion with SIM(3) Equivariance](https://mlsanity.com/paper/2509.26631/) (2025)
- [Adapting Segment Anything Model for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2409.15481/) (2024)
- [Implicit Contact Diffuser: Sequential Contact Reasoning with Latent Point Cloud Diffusion](https://mlsanity.com/paper/2410.16571/) (2024)
- [CLASP: General-Purpose Clothes Manipulation with Semantic Keypoints](https://mlsanity.com/paper/2507.19983/) (2025)
- [Efficient Hybrid SE(3)-Equivariant Visuomotor Flow Policy via Spherical Harmonics for Robot Manipulation](https://mlsanity.com/paper/2603.23227/) (2026)
- [The Embodiment Gap in Robot Foundation Models](https://mlsanity.com/paper/2608.18433/) (2026)

## Abstract (arXiv)

If a robot masters folding a kitchen towel, we would expect it to master folding a large beach towel. However, existing policy learning methods that rely on data augmentation still don't guarantee such generalization. Our insight is to add equivariance to both the visual object representation and policy architecture. We propose EquivAct which utilizes SIM(3)-equivariant network structures that guarantee generalization across all possible object translations, 3D rotations, and scales by construction. EquivAct is trained in two phases. We first pre-train a SIM(3)-equivariant visual representation on simulated scene point clouds. Then, we learn a SIM(3)-equivariant visuomotor policy using a small amount of source task demonstrations. We show that the learned policy directly transfers to objects that substantially differ from demonstrations in scale, position, and orientation. We evaluate our method in three manipulation tasks involving deformable and articulated objects, going beyond typical rigid object manipulation tasks considered in prior work. We conduct experiments both in simulation and in reality. For real robot experiments, our method uses 20 human demonstrations of a tabletop task and transfers zero-shot to a mobile manipulation task in a much larger setup. Experiments confirm that our contrastive pre-training procedure and equivariant architecture offer significant improvements over prior work. Project website: https://equivact.github.io