# GarmentAligner: Text-to-Garment Generation via Retrieval-augmented Multi-level Corrections arXiv: https://arxiv.org/abs/2408.12352 · HTML: https://mlsanity.com/paper/2408.12352/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. - Builds on 28 of its 44 references that are in the map. - First posted on arXiv 22 Aug 2024. ## Placed in - Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/) ## Builds on (most cited first, 28 of 28) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020) - [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022) - [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021) - [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023) - [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024) - [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023) - [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019) - [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023) - [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022) - [GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium](https://mlsanity.com/paper/1706.08500/) (2017) - [Re-Imagen: Retrieval-Augmented Text-to-Image Generator](https://mlsanity.com/paper/2209.14491/) (2022) - [Any-to-Any Generation via Composable Diffusion](https://mlsanity.com/paper/2305.11846/) (2023) - [RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths](https://mlsanity.com/paper/2305.18295/) (2023) - [Retrieval-Augmented Multimodal Language Modeling](https://mlsanity.com/paper/2211.12561/) (2022) - [Semi-Parametric Neural Image Synthesis](https://mlsanity.com/paper/2204.11824/) (2022) - [HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models](https://mlsanity.com/paper/2401.07450/) (2024) - [Quality and Quantity: Unveiling a Million High-Quality Images for Text-to-Image Synthesis in Fashion Design](https://mlsanity.com/paper/2311.12067/) (2023) - [RAFIC: Retrieval-Augmented Few-shot Image Classification](https://mlsanity.com/paper/2312.06868/) (2023) ## Built on by (most cited first, 10 of 10) - [Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook](https://mlsanity.com/paper/2503.18016/) (2025) - [RAGDiffusion: Faithful Cloth Generation via External Knowledge Assimilation](https://mlsanity.com/paper/2411.19528/) (2024) - [MGT: Extending Virtual Try-Off to Multi-Garment Scenarios](https://mlsanity.com/paper/2504.13078/) (2025) - [IGR: Improving Diffusion Model for Garment Restoration from Person Image](https://mlsanity.com/paper/2412.11513/) (2024) - [HiGarment: Cross-modal Harmony Based Diffusion Model for Flat Sketch to Realistic Garment Image](https://mlsanity.com/paper/2505.23186/) (2025) - [QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding](https://mlsanity.com/paper/2601.18195/) (2026) - [EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluation](https://mlsanity.com/paper/2508.03497/) (2025) - [OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework](https://mlsanity.com/paper/2603.19643/) (2026) - [Multi-Level Conditioning by Pairing Localized Text and Sketch for Fashion Image Generation](https://mlsanity.com/paper/2602.18309/) (2026) - [AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interaction](https://mlsanity.com/paper/2603.05551/) (2026) ## Abstract (arXiv) General text-to-image models bring revolutionary innovation to the fields of arts, design, and media. However, when applied to garment generation, even the state-of-the-art text-to-image models suffer from fine-grained semantic misalignment, particularly concerning the quantity, position, and interrelations of garment components. Addressing this, we propose GarmentAligner, a text-to-garment diffusion model trained with retrieval-augmented multi-level corrections. To achieve semantic alignment at the component level, we introduce an automatic component extraction pipeline to obtain spatial and quantitative information of garment components from corresponding images and captions. Subsequently, to exploit component relationships within the garment images, we construct retrieval subsets for each garment by retrieval augmentation based on component-level similarity ranking and conduct contrastive learning to enhance the model perception of components from positive and negative samples. To further enhance the alignment of components across semantic, spatial, and quantitative granularities, we propose the utilization of multi-level correction losses that leverage detailed component information. The experimental findings demonstrate that GarmentAligner achieves superior fidelity and fine-grained semantic alignment when compared to existing competitors.