mlsanityData to 14 Oct 2026

Inverse Garment Design with Visual Agents

arXiv 2610.15582 ↗First posted 11 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We study the problem of inverse garment design, reconstructing simulation-ready 3D outfits with arbitrary layers from a single image, with visual agents. Unlike differentiable optimization that is time-consuming, or feed-forward methods limited by training data, we introduce DressAgent, a hierarchical agentic system that recovers garment sewing patterns through planning and closed-loop refinement grounded in physical simulation. To this end, we first present an enhanced representation based on GarmentCode for diverse styles and arbitrary dressing layers, and build the first multi-layer sewing pattern dataset with over 15K outfits for supervision and benchmarking. Inverse design then proceeds coarse to fine: In layer planning and coarse estimation, agents infer layer order and relations, peel each layer into its own view, and estimate coarse patterns. In fine-grained refinement, an agentic trio of Tailor, Critic, and Orchestrator refines patterns in a closed loop of editing, simulating, and judging against the reference with visual and measured evidence. Carefully designed tools and an evolving memory make this refinement converge on the target geometry. We also design an agentic generation method for high-fidelity textures. DressAgent outperforms feed-forward methods by a large margin, surpasses or matches differentiable optimization in a fraction of the time, and generalizes to complex layering, diverse poses, and body shapes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 56 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

40 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning16 Jan

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  11. 2024TextGrad: Automatic "Differentiation" via Textcited by 273
  12. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
27 more
  1. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 155
  2. 2020TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Stylecited by 149
  3. 2019DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Imagescited by 134
  4. 2020BCNet: Learning Body and Cloth Shape from A Single Imagecited by 120
  5. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  6. 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 59
  7. 20244D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotationscited by 54
  8. 2024DressCode: Autoregressively Sewing and Generating Garments from Text Guidancecited by 44
  9. 20223D Clothed Human Reconstruction in the Wildcited by 42
  10. 2022DrapeNet: Garment Generation and Self-Supervised Drapingcited by 41
  11. 2023DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulationcited by 41
  12. 2023Towards Garment Sewing Pattern Reconstruction from a Single Imagecited by 34
  13. 2023ISP: Multi-Layered Garment Draping with Implicit Sewing Patternscited by 32
  14. 2024GarmentCodeData: A Dataset of 3D Made-to-Measure Garments With Sewing Patternscited by 28
  15. 2023GarmentCode: Programming Parametric Sewing Patternscited by 27
  16. 2023Garment Recovery with Shape and Deformation Priorscited by 24
  17. 2024ChatGarment: Garment Estimation, Generation and Editing via Large Language Modelscited by 23
  18. 2024CloSe: A 3D Clothing Segmentation Dataset and Modelcited by 22
  19. 2025Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physicscited by 20
  20. 2024Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesiscited by 18
  21. 2024GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Detailscited by 16
  22. 2024A Comparative Study of DSL Code Generation: Fine-Tuning vs. Optimized Retrieval Augmentationcited by 10
  23. 2026Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agentscited by 4
  24. 2025RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Taskscited by 2
  25. 2026DressWild: Feed-Forward Pose-Agnostic Garment Sewing Pattern Generation from In-the-Wild Imagescited by 2
  26. 2026NGL: Natural Garment Language for Training-Free Sewing Pattern Estimationcited by 2
  27. 2026Enhancing Hands in 3D Whole-Body Pose Estimation with Conditional Hands Modulatorcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PatternGSL: A Structured Specification Language for Template-Free and Simulation-Ready 3D Garments
  2. 2025GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling
  3. 2025Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates
  4. 2025GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing
  5. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details
  6. 2026Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design
  7. 2026Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing
  8. 2026Image2Garment: Simulation-ready Garment Generation from a Single Image