# FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL

arXiv: https://arxiv.org/abs/2506.05501 · HTML: https://mlsanity.com/paper/2506.05501/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 25 of 25)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation](https://mlsanity.com/paper/2404.14396/) (2024)
- [Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis](https://mlsanity.com/paper/2412.04431/) (2024)
- [AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea](https://mlsanity.com/paper/2411.15738/) (2024)
- [World Model on Million-Length Video And Language With Blockwise RingAttention](https://mlsanity.com/paper/2402.08268/) (2024)
- [Making LLaMA SEE and Draw with SEED Tokenizer](https://mlsanity.com/paper/2310.01218/) (2023)
- [T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT](https://mlsanity.com/paper/2505.00703/) (2025)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [Auto-Encoding Morph-Tokens for Multimodal LLM](https://mlsanity.com/paper/2405.01926/) (2024)
- [Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens](https://mlsanity.com/paper/2504.14666/) (2025)
- [STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training](https://mlsanity.com/paper/2412.00161/) (2024)
- [SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs](https://mlsanity.com/paper/2408.11813/) (2024)
- [Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration](https://mlsanity.com/paper/2409.19872/) (2024)
- [Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization](https://mlsanity.com/paper/2303.12314/) (2023)
- [Exploring Bias in over 100 Text-to-Image Generative Models](https://mlsanity.com/paper/2503.08012/) (2025)
- [Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning](https://mlsanity.com/paper/2506.01480/) (2025)
- [Improving Vision Anomaly Detection with the Guidance of Language Modality](https://mlsanity.com/paper/2310.02821/) (2023)
- [Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge](https://mlsanity.com/paper/2411.16824/) (2024)

## Built on by (most cited first, 13 of 13)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Group Critical-token Policy Optimization for Autoregressive Image Generation](https://mlsanity.com/paper/2509.22485/) (2025)
- [STAGE: Stable and Generalizable GRPO for Autoregressive Image Generation](https://mlsanity.com/paper/2509.25027/) (2025)
- [DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2512.05112/) (2025)
- [WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing](https://mlsanity.com/paper/2512.00387/) (2025)
- [Stepwise Credit Assignment for GRPO on Flow-Matching Models](https://mlsanity.com/paper/2603.28718/) (2026)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoning](https://mlsanity.com/paper/2509.22761/) (2025)
- [CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2603.08652/) (2026)
- [FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation](https://mlsanity.com/paper/2604.13491/) (2026)
- [Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs](https://mlsanity.com/paper/2509.05714/) (2025)
- [OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructions](https://mlsanity.com/paper/2512.19159/) (2025)
- [From "What" to "How": Constrained Reasoning for Autoregressive Image Generation](https://mlsanity.com/paper/2603.02712/) (2026)

## Abstract (arXiv)

Recent studies extend the autoregression paradigm to text-to-image generation, achieving performance comparable to diffusion models. However, our new PairComp benchmark -- featuring test cases of paired prompts with similar syntax but different fine-grained semantics -- reveals that existing models struggle with fine-grained text-image alignment thus failing to realize precise control over visual tokens. To address this, we propose FocusDiff, which enhances fine-grained text-image semantic alignment by focusing on subtle differences between similar text-image pairs. We construct a new dataset of paired texts and images with similar overall expressions but distinct local semantics, further introducing a novel reinforcement learning algorithm to emphasize such fine-grained semantic differences for desired image generation. Our approach achieves state-of-the-art performance on existing text-to-image benchmarks and significantly outperforms prior methods on PairComp.