GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
Abstract (arXiv)
Text logo design heavily relies on the creativity and expertise of professional designers, in which arranging element layouts is one of the most important procedures. However, this specific task has received limited attention, often overshadowed by broader layout generation tasks such as document or poster design. In this paper, we propose a Vision-Language Model (VLM)-based framework that generates content-aware text logo layouts by integrating multi-modal inputs with user-defined constraints, enabling more flexible and robust layout generation for real-world applications. We introduce two model techniques that reduce the computational cost for processing multiple glyph images simultaneously, without compromising performance. To support instruction tuning of our model, we construct two extensive text logo datasets that are five times larger than existing public datasets. In addition to geometric annotations (textite.g., text masks and character recognition), our datasets include detailed layout descriptions in natural language, enabling the model to reason more effectively in handling complex designs and custom user inputs. Experimental results demonstrate the effectiveness of our proposed framework and datasets, outperforming existing methods on various benchmarks that assess geometric aesthetics and human preferences.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 46 references that are in the map. First posted on arXiv 18 Nov 2024.
Builds on
39 of 46 references.
Closest ancestors · 1
Same direction, newest first
Other references · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Visual Instruction Tuningcited by 7,869
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2016Improved Techniques for Training GANscited by 2,970
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
26 more
- 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
- 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
- 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
- 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
- 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
- 2021Variational Transformer Networks for Layout Generationcited by 88
- 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
- 2024DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Modelscited by 75
- 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
- 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 61
- 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
- 2018Analogical Reasoning on Chinese Morphological and Semantic Relationscited by 50
- 2023LayoutPrompter: Awaken the Design Ability of Large Language Modelscited by 48
- 2022LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restrictioncited by 43
- 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
- 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
- 2024PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLMcited by 41
- 2022Geometry Aligned Variational Transformer for Image-conditioned Layout Generationcited by 33
- 2022Aesthetic Text Logo Synthesis via Content-aware Layout Inferringcited by 19
- 2022LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designercited by 15
- 2023A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptionscited by 15
- 2023WordArt Designer: User-Driven Artistic Typography Synthesis using Large Language Modelscited by 15
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026LogoDiffuser: Training-Free Multilingual Logo Generation and Stylization via Letter-Aware Attention Control
- 2024GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models
- 2026VecGlypher: Unified Vector Glyph Generation with Language Models
- 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models
- 2026LoGAN: Multilingual Font Localization with Generative Agents
- 2023GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
- 2025Automatic Text Box Placement for Supporting Typographic Design
- 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis