# FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing

arXiv: https://arxiv.org/abs/2509.23452 · HTML: https://mlsanity.com/paper/2509.23452/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 98 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 30 of 32)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Scaling Open-Vocabulary Object Detection](https://mlsanity.com/paper/2306.09683/) (2023)
- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.13655/) (2023)
- [MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation](https://mlsanity.com/paper/2307.14460/) (2023)
- [Self-correcting LLM-controlled Diffusion Models](https://mlsanity.com/paper/2311.16090/) (2023)
- [FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition](https://mlsanity.com/paper/2312.07536/) (2023)
- [Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models](https://mlsanity.com/paper/2305.13921/) (2023)
- [SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoning](https://mlsanity.com/paper/2104.05832/) (2021)
- [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://mlsanity.com/paper/2204.08292/) (2022)
- [Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities](https://mlsanity.com/paper/2410.17385/) (2024)
- [DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback](https://mlsanity.com/paper/2311.17946/) (2023)
- [Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models](https://mlsanity.com/paper/2412.18605/) (2024)
- [Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models](https://mlsanity.com/paper/2502.08636/) (2025)
- [Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning](https://mlsanity.com/paper/2210.16952/) (2022)
- [SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models](https://mlsanity.com/paper/2410.03878/) (2024)
- [AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation](https://mlsanity.com/paper/2406.05000/) (2024)
- [FoREST: Frame of Reference Evaluation in Spatial Reasoning Tasks](https://mlsanity.com/paper/2502.17775/) (2025)
- [GenSpace: Benchmarking Spatially-Aware Image Generation](https://mlsanity.com/paper/2505.24870/) (2025)

## Built on by (most cited first, 1 of 1)

- [SATURN: Symbolic Spatial Reasoning for Multi-Perspective Grounding](https://mlsanity.com/paper/2606.22694/) (2026)

## Abstract (arXiv)

Current text-to-image generation models, even state-of-the-art models, exhibit a significant performance gap when spatial expressions are described from non-camera perspectives. To address this limitation, we propose Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing (FoR-SALE), an extension of the Self-correcting LLM-controlled Diffusion (SLD). FoR-SALE first evaluates the alignment between a given text and an initially generated image, and then refines the image based on the expressed FoR in the spatial description. It employs vision modules to extract the spatial configuration of the generated image and simultaneously maps the spatial expression to a corresponding camera perspective. This unified perspective enables direct evaluation of alignment between language and vision. When misalignment is detected, the required editing operations are generated and applied. FoR-SALE introduces novel latent-space operations to adjust the facing direction and depth of generated images. We evaluate FoR-SALE on three benchmarks designed to assess spatial understanding with FoR: FoR-LMD, FoREST, and FoREST-G, with the latter providing diagnostic subsets covering multi-object scenes, expanded relation types, and naturalized prompts to test the generality of our framework. Our framework improves the performance of SOTA T2I models by up to 7.0 pp using only a single round of correction.