mlsanityData to 14 Oct 2026

Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework

arXiv 2606.19073 ↗First posted 17 Jun 2026cs.CVcited by 5

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Current image editing methods excel at static attributes but fail at complex Human-Object Interactions (HOI), a critical challenge unaddressed by existing benchmarks that conflate HOI with static attributes, relying on global metrics incapable of simultaneously assessing dynamic interaction validity and entangled human-object pair preservation. Thus, we first introduce HOI-Edit, a comprehensive benchmark with three progressive cognitive levels, which features an automated metric HOI-Eval that reliably evaluates instance-level interaction by letting VLM Q&A after thinking with images containing grounded Human-Object pairs. Considering the task's essence of remodeling dynamic relationships, we benchmark Image-to-Video (I2V) models, finding them inherently suited for dynamic editing due to their temporal generation capabilities. Crucially, beyond superior performance, this capability provides a "replay of the failure process," offering unique diagnosability into why errors occur. We thus propose SCPE (Self-Correcting Process Editing), a novel, agentic self-correcting framework that constrains the generation of I2V models through iteratively refined prompts, enabling the generated videos to more accurately present the target HOI. Extracted frames from these videos are the final editing results. On HOI-Edit, SCPE achieves performance competitive with state-of-the-art (SOTA) editing models like Nano Banana on interaction. Code is available at https://github.com/oceanflowlab/HOI-Edit.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 44 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

33 of 44 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  7. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  8. 2025Qwen-Image Technical Reportcited by 1,044
  9. 2025OpenAI GPT-5 System Cardcited by 989
  10. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  11. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  12. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
21 more
  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  2. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  3. 2017Learning to Detect Human-Object Interactionscited by 240
  4. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  5. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 204
  6. 2026Qwen3.5-Omni Technical Reportcited by 174
  7. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 101
  8. 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 63
  9. 2023UniWorld: Autonomous Driving Pre-training via World Modelscited by 33
  10. 2025TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Timescited by 33
  11. 2025ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulationcited by 29
  12. 2025HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generationcited by 26
  13. 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
  14. 2025LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMscited by 24
  15. 2025ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motionscited by 19
  16. 2025PICABench: How Far Are We from Physically Realistic Image Editing?cited by 16
  17. 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
  18. 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11
  19. 2025Balancing Preservation and Modification: A Region and Semantic Aware Metric for Instruction-Based Image Editingcited by 10
  20. 2025Open-Vocabulary HOI Detection with Interaction-aware Prompt and Concept Calibrationcited by 8
  21. 2025Interact-Custom: Customized Human Object Interaction Image Generationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025I2I-Bench: A Comprehensive Benchmark Suite for Image-to-Image Editing Models
  2. 2026OneHOI: Unifying Human-Object Interaction Generation and Editing
  3. 2026InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinement
  4. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?
  5. 2026I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing
  6. 2026EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement
  7. 2026ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
  8. 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editing