mlsanityData to 14 Oct 2026

Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement

arXiv 2603.22187 ↗First posted 23 Mar 2026cs.CV cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

Recent advances in Multimodal Large Language Models (MLLMs) have enabled automated generation of structured layouts from natural language descriptions. Existing methods typically follow a code-only paradigm that generates code to represent layouts, which are then rendered by graphic engines to produce final images. However, they are blind to the rendered visual outcome, making it difficult to guarantee readability and aesthetics. In this paper, we identify visual feedback as a critical factor in layout generation and propose Visual Feedback Layout Model (VFLM), a self-improving framework that leverages visual feedback iterative refinement. VFLM is capable of performing adaptive reflective generation, which leverages visual information to reflect on previous issues and iteratively generates outputs until satisfactory quality is achieved. It is achieved through reinforcement learning with a visually grounded reward model that incorporates OCR accuracy. By rewarding only the final generated outcome, we can effectively stimulate the model's iterative and reflective generative capabilities. Experiments across multiple benchmarks show that VFLM consistently outperforms advanced MLLMs, existing layout models, and code-only baselines, establishing visual feedback as critical for design-oriented MLLMs. Our code and data are available at https://github.com/FolSpark/VFLM.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 56 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

48 of 56 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2025Qwen3 Technical Reportcited by 8,820
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2025Qwen2.5-VL Technical Reportcited by 5,587
  12. 2024GPT-4o System Cardcited by 4,588
36 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  8. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  9. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  10. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  11. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  12. 2025Seed1.5-VL Technical Reportcited by 373
  13. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  14. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  15. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  16. 2025Agentic Reinforced Policy Optimizationcited by 172
  17. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 110
  18. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 95
  19. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  20. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 71
  21. 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
  22. 2023LayoutPrompter: Awaken the Design Ability of Large Language Modelscited by 48
  23. 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 43
  24. 2023AutoPoster: A Highly Automatic and Content-aware Design System for Advertising Poster Generationcited by 41
  25. 2024PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLMcited by 41
  26. 2024Graphic Design with Large Multimodal Modelcited by 40
  27. 2024OpenCOLE: Towards Reproducible Automatic Graphic Design Generationcited by 37
  28. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
  29. 2023Towards Flexible Multi-modal Document Modelscited by 31
  30. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 16
  31. 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 15
  32. 2024Can GPTs Evaluate Graphic Design Based on Design Principles?cited by 14
  33. 2025SVGen: Interpretable Vector Graphics Generation with Large Language Modelscited by 14
  34. 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
  35. 2024TextLap: Customizing Language Models for Text-to-Layout Planningcited by 8
  36. 2025Smaller But Better: Unifying Layout Generation with Smaller Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documentscited by 1
  2. 2026CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learningcited by 0
  3. 2026One Figure, Every Canvas: Editable Flowchart Relayout via Agentic Pipelinecited by 0
  4. 2026PLAYGROUND: Progressive Layout Generation with Render-Grounded Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Prompting with Iterative Refinement for Design Critique Generation
  2. 2024Leveraging Human Revisions for Improving Text-to-Layout Models
  3. 2023Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis
  4. 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluation
  5. 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models
  6. 2026From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design
  7. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering
  8. 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design