mlsanityData to 7 Oct 2026

I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing

arXiv 2601.03741 ↗First posted 7 Jan 2026cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Existing text-guided image editing methods primarily rely on end-to-end pixel-level inpainting paradigm. Despite its success in simple scenarios, this paradigm still significantly struggles with compositional editing tasks that require precise local control and complex multi-object spatial reasoning. This paradigm is severely limited by 1) the implicit coupling of planning and execution, 2) the lack of object-level control granularity, and 3) the reliance on unstructured, pixel-centric modeling. To address these limitations, we propose I2E, a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions into a series of atomic actions via Chain-of-Thought reasoning. Further, we also construct I2E-Bench, a benchmark designed for multi-instance spatial reasoning and high-precision editing. Experimental results on I2E-Bench and multiple public benchmarks demonstrate that I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

26 of 45 references.

References · 26

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2018World Modelscited by 1,444
  10. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  11. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  12. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
14 more
  1. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  2. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  3. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  4. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
  5. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 100
  6. 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
  7. 2025GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Datasetcited by 68
  8. 2025OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Datacited by 35
  9. 2021Instance-wise Occlusion and Depth Orders in Natural Scenescited by 31
  10. 2025Alignment for Efficient Tool Calling of Large Language Modelscited by 17
  11. 2025Image Editing As Programs with Diffusion Modelscited by 10
  12. 2024Addressing Text Embedding Leakage in Diffusion-based Image Editingcited by 7
  13. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineeringcited by 7
  14. 2025MPR-GUI: Benchmarking and Enhancing Multilingual Perception and Reasoning in GUI Agentscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026IMAGAgent: Orchestrating Multi-Turn Image Editing via Constraint-Aware Planning and Reflection
  2. 2026Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Tooling
  3. 2026Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
  4. 2026PhysEdit: Physically-Consistent Region-Aware Image Editing via Adaptive Spatio-Temporal Reasoning
  5. 2025ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies
  6. 2025CompBench: Benchmarking Complex Instruction-guided Image Editing
  7. 2026MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance
  8. 2026Rethinking Where to Edit: Task-Aware Localization for Instruction-Based Image Editing