# TransPhy: Visual In-Context Learning for Physically Grounded Image Editing

arXiv: https://arxiv.org/abs/2608.24119 · HTML: https://mlsanity.com/paper/2608.24119/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts.
- Builds on 31 of its 36 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Placed in

- Mixture-of-experts / [Multimodal MoE models](https://mlsanity.com/moe/multimodal-moe-models/)

## Builds on (most cited first, 30 of 31)

- [Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2410.13848/) (2024)
- [In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer](https://mlsanity.com/paper/2504.20690/) (2025)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [DiT4Edit: Diffusion Transformer for Image Editing](https://mlsanity.com/paper/2411.03286/) (2024)
- [Insert Anything: Image Insertion via In-Context Editing in DiT](https://mlsanity.com/paper/2504.15009/) (2025)
- [ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation](https://mlsanity.com/paper/2308.00906/) (2023)
- [VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning](https://mlsanity.com/paper/2504.07960/) (2025)
- [Customizing Text-to-Image Models with a Single Image Pair](https://mlsanity.com/paper/2405.01536/) (2024)
- [Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model](https://mlsanity.com/paper/2405.10316/) (2024)
- [ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation](https://mlsanity.com/paper/2510.04290/) (2025)
- [Visual Instruction Inversion: Image Editing via Visual Prompting](https://mlsanity.com/paper/2307.14331/) (2023)
- [RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers](https://mlsanity.com/paper/2506.02528/) (2025)
- [X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learning](https://mlsanity.com/paper/2508.07607/) (2025)
- [UniREditBench: A Unified Reasoning-based Image Editing Benchmark](https://mlsanity.com/paper/2511.01295/) (2025)
- [Edit Transfer: Learning Image Editing via Vision In-Context Relations](https://mlsanity.com/paper/2503.13327/) (2025)
- [AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning](https://mlsanity.com/paper/2409.00263/) (2024)
- [PICABench: How Far Are We from Physically Realistic Image Editing?](https://mlsanity.com/paper/2510.17681/) (2025)
- [An Item is Worth a Prompt: Versatile Image Editing with Disentangled Control](https://mlsanity.com/paper/2403.04880/) (2024)
- [PairEdit: Learning Semantic Variations for Exemplar-based Image Editing](https://mlsanity.com/paper/2506.07992/) (2025)
- [LoRA of Change: Learning to Generate LoRA for the Editing Instruction from A Single Before-After Image Pair](https://mlsanity.com/paper/2411.19156/) (2024)
- [PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery](https://mlsanity.com/paper/2501.09826/) (2025)
- [Spanning the Visual Analogy Space with a Weight Basis of LoRAs](https://mlsanity.com/paper/2602.15727/) (2026)
- [TweezeEdit: Consistent and Efficient Image Editing with Path Regularization](https://mlsanity.com/paper/2508.10498/) (2025)
- [VIRAL: Visual In-Context Reasoning via Analogy in Diffusion Transformers](https://mlsanity.com/paper/2602.03210/) (2026)
- [WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark](https://mlsanity.com/paper/2602.07095/) (2026)
- [From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priors](https://mlsanity.com/paper/2602.21778/) (2026)
- [Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing](https://mlsanity.com/paper/2606.05172/) (2026)
- [PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing](https://mlsanity.com/paper/2606.26551/) (2026)
- [FunEditor: Achieving Complex Image Edits via Function Aggregation with Diffusion Models](https://mlsanity.com/paper/2408.08495/) (2024)
- [InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models](https://mlsanity.com/paper/2603.03657/) (2026)

## Abstract (arXiv)

Visual demonstrations provide a natural interface for specifying image transformations that are difficult to describe exhaustively with text. However, existing visual in-context learning (VICL) methods primarily focus on appearance-level relation transfer and provide limited support for physically grounded transformations, whose outcomes depend on material properties, geometry, object interactions, and environmental conditions. Given a source--target exemplar pair and a query image, physically grounded VICL requires a model to infer the demonstrated transformation, adapt its effects to the query-specific scene context, and preserve rule-irrelevant content. We introduce PhysVICL-74, comprising 74 physically grounded transformation rules and 5,240 source--target image pairs that form nearly 75K training and evaluation contexts. Its benchmark split separately evaluates novel-instance transfer and unseen-rule generalization. We further propose TransPhy, a framework that decomposes physically grounded VICL into physical-rule induction and transition-aligned rendering. TransPhy first predicts the demonstrated rule and an explicit query-specific target-state description, and then synthesizes the target image through token-wise mixture-of-experts adaptation, with expert routing guided by localized transition cues. Experiments show that TransPhy improves physical-rule adherence, query consistency, and unseen-rule generalization over existing visual in-context editing methods.