# Geometrical Properties of Text Token Embeddings for Strong Semantic Binding in Text-to-Image Generation

arXiv: https://arxiv.org/abs/2503.23011 · HTML: https://mlsanity.com/paper/2503.23011/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 53 references that are in the map.
- First posted on arXiv 29 Mar 2025.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [Optimizing Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2212.09611/) (2022)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment](https://mlsanity.com/paper/2306.08877/) (2023)
- [Grounded Text-to-Image Synthesis with Attention Refocusing](https://mlsanity.com/paper/2306.05427/) (2023)
- [Dense Text-to-Image Generation with Attention Modulation](https://mlsanity.com/paper/2308.12964/) (2023)
- [Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT](https://mlsanity.com/paper/2406.18583/) (2024)
- [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models](https://mlsanity.com/paper/2305.13921/) (2023)
- [Divide & Bind Your Attention for Improved Generative Semantic Nursing](https://mlsanity.com/paper/2307.10864/) (2023)
- [Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following](https://mlsanity.com/paper/2311.17002/) (2023)
- [Directed Diffusion: Direct Control of Object Placement through Attention Guidance](https://mlsanity.com/paper/2302.13153/) (2023)

## Built on by (most cited first, 5 of 5)

- [Training-free Mixed-Resolution Latent Upsampling for Spatially Accelerated Diffusion Transformers](https://mlsanity.com/paper/2507.08422/) (2025)
- [Rethinking Global Text Conditioning in Diffusion Transformers](https://mlsanity.com/paper/2602.09268/) (2026)
- [Formalizing the Binding Problem](https://mlsanity.com/paper/2606.03976/) (2026)
- [STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model](https://mlsanity.com/paper/2606.10653/) (2026)
- [Intermediate Text Representation Guided Text-to-Image Generation for Enhancing One-and-Only Alignment](https://mlsanity.com/paper/2606.30262/) (2026)

## Abstract (arXiv)

Text-to-image (T2I) models often suffer from text-image misalignment in complex scenes involving multiple objects and attributes. Semantic binding has attempted to associate the generated attributes and objects with their corresponding noun phrases (NPs) by text or latent optimizations with the modulation of cross-attention (CA) maps; yet, the factors that influence semantic binding remain underexplored. Here, we investigate the geometrical properties of text token embeddings and their CA maps. We found that the geometrical properties of token embeddings, specifically angular distances and norms, are crucial factors in the differentiation of the CA map. These theoretical findings led to our proposed training-free text-embedding-aware T2I framework, dubbed TokeBi, for strong semantic binding. TokeBi consists of Causality-Aware Projection-Out (CAPO) for distinguishing inter-NP CA maps and Adaptive Token Mixing (ATM) for enhancing inter-NP separation while maintaining intra-NP cohesion in CA maps. Extensive experiments confirm that TokeBi outperforms prior arts across diverse baselines and datasets.