# Copy-Trasform-Paste: Zero-Shot Object-Object Alignment Guided by Vision-Language and Geometric Constraints

arXiv: https://arxiv.org/abs/2601.14207 · HTML: https://mlsanity.com/paper/2601.14207/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [ContactOpt: Optimizing Contact to Improve Grasps](https://mlsanity.com/paper/2104.07267/) (2021)
- [TextDeformer: Geometry Manipulation using Text Guidance](https://mlsanity.com/paper/2304.13348/) (2023)
- [SceneTeller: Language-to-3D Scene Generation](https://mlsanity.com/paper/2407.20727/) (2024)
- [CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics](https://mlsanity.com/paper/2212.02122/) (2022)
- [PrEditor3D: Fast and Precise 3D Shape Editing](https://mlsanity.com/paper/2412.06592/) (2024)
- [SceneMotifCoder: Example-driven Visual Program Learning for Generating 3D Object Arrangements](https://mlsanity.com/paper/2408.02211/) (2024)
- [CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation](https://mlsanity.com/paper/2502.19842/) (2025)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [Text-guided Controllable Mesh Refinement for Interactive 3D Modeling](https://mlsanity.com/paper/2406.01592/) (2024)
- [Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2503.19914/) (2025)

## Built on by (most cited first, 1 of 1)

- [Interact3D: Compositional 3D Generation of Interactive Objects](https://mlsanity.com/paper/2603.16085/) (2026)

## Abstract (arXiv)

We study zero-shot 3D alignment of two given meshes, using a text prompt describing their spatial relation -- an essential capability for content creation and scene assembly. Earlier approaches primarily rely on geometric alignment procedures, while recent work leverages pretrained 2D diffusion models to model language-conditioned object-object spatial relationships. In contrast, we directly optimize the relative pose at test time, updating translation, rotation, and isotropic scale with CLIP-driven gradients via a differentiable renderer, without training a new model. Our framework augments language supervision with geometry-aware objectives: a variant of soft-Iterative Closest Point (ICP) term to encourage surface attachment and a penetration loss to discourage interpenetration. A phased schedule strengthens contact constraints over time, and camera control concentrates the optimization on the interaction region. To enable evaluation, we curate a benchmark containing diverse categories and relations, and compare against baselines. Our method outperforms all alternatives, yielding semantically faithful and physically plausible alignments.