mlsanityData to 7 Oct 2026

CtrlVTON: Controllable Virtual Try-On via Visual-Instance-Prompt Segmentation

arXiv 2607.09362 ↗First posted 10 Jul 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Virtual try-on (VTO) has made significant progress in realistically transferring garments onto a target person. Yet most systems give the user little control over how a garment should be worn -- its size (loose or fitted), style (e.g., tucked in or untucked, open or closed), and spatial placement on the body. We address this gap with two complementary contributions. First, we define and solve Visual-Instance-Prompt Segmentation via VIP-SAM: given a flatlay image of a garment, segment that specific instance in a photograph of a person wearing it. This is an instance-level task, distinct from the typically studied category-level segmentation. Second, we introduce CtrlVTON, a controllable VTO framework that recasts try-on as an image editing problem and adds segmentation masks as pixel-level control over garment layout, including style, size, and spatial placement on the body. VIP-SAM and CtrlVTON each achieve state-of-the-art results on their respective tasks. In particular, CtrlVTON generates images that follow user-provided layouts far more faithfully than the strongest proprietary editing systems while matching them on garment fidelity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 64 of its 74 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

64 of 74 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2023Segment Anythingcited by 8,158
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  10. 2025Qwen3-VL Technical Reportcited by 2,534
  11. 2025DINOv3cited by 1,297
  12. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
38 more
  1. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  2. 2025SAM 3: Segment Anything with Conceptscited by 835
  3. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  4. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  5. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  6. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  7. 2017One-Shot Learning for Semantic Segmentationcited by 337
  8. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  9. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  10. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  11. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  12. 2017VITON: An Image-based Virtual Try-on Networkcited by 285
  13. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 276
  14. 2023Personalize Segment Anything Model with One Shotcited by 250
  15. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  16. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  17. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
  18. 2018Toward Characteristic-Preserving Image-based Virtual Try-On Networkcited by 192
  19. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  20. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  21. 2023StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-Oncited by 173
  22. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 168
  23. 2024In-Context LoRA for Diffusion Transformerscited by 155
  24. 2023LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-Oncited by 151
  25. 2020Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Contentcited by 148
  26. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  27. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  28. 2023TryOnDiffusion: A Tale of Two UNetscited by 141
  29. 2022High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditionscited by 129
  30. 2023GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learningcited by 120
  31. 2024Improving Diffusion Models for Authentic Virtual Try-on in the Wildcited by 109
  32. 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
  33. 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
  34. 2025Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Taskscited by 62
  35. 2024FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-oncited by 39
  36. 2024M&M VTO: Multi-Garment Virtual Try-On and Editingcited by 29
  37. 2023Differential Diffusion: Giving Each Pixel Its Strengthcited by 27
  38. 2024Learning Flow Fields in Attention for Controllable Person Image Generationcited by 27

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.