# Untwisting RoPE: Frequency Control for Shared Attention in DiTs

arXiv: https://arxiv.org/abs/2602.05013 · HTML: https://mlsanity.com/paper/2602.05013/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [Taming Rectified Flow for Inversion and Editing](https://mlsanity.com/paper/2411.04746/) (2024)
- [AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers](https://mlsanity.com/paper/2411.18673/) (2024)
- [Cross-Image Attention for Zero-Shot Appearance Transfer](https://mlsanity.com/paper/2311.03335/) (2023)
- [Implicit Style-Content Separation using B-LoRA](https://mlsanity.com/paper/2403.14572/) (2024)
- [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024)
- [Inversion-Free Image Editing with Natural Language](https://mlsanity.com/paper/2312.04965/) (2023)
- [StyleMaster: Stylize Your Video with Artistic Generation and Translation](https://mlsanity.com/paper/2412.07744/) (2024)
- [FreeFlux: Understanding and Exploiting Layer-Specific Roles in RoPE-Based MMDiT for Versatile Image Editing](https://mlsanity.com/paper/2503.16153/) (2025)
- [Consolidating Attention Features for Multi-view Image Editing](https://mlsanity.com/paper/2402.14792/) (2024)
- [Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation](https://mlsanity.com/paper/2408.14819/) (2024)
- [DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion](https://mlsanity.com/paper/2510.20766/) (2025)
- [Z-STAR+: A Zero-shot Style Transfer Method via Adjusting Style Distribution](https://mlsanity.com/paper/2411.19231/) (2024)
- [The Devil is in Attention Sharing: Improving Complex Non-rigid Image Editing Faithfulness via Attention Synergy](https://mlsanity.com/paper/2512.14423/) (2025)
- [Griffin: Generative Reference and Layout Guided Image Composition](https://mlsanity.com/paper/2509.23643/) (2025)

## Built on by (most cited first, 5 of 5)

- [FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining](https://mlsanity.com/paper/2606.20506/) (2026)
- [StyleComposer: Training-Free Multi-Reference Style Composition](https://mlsanity.com/paper/2608.05213/) (2026)
- [An Evaluation Framework for Generating Multi-View Images of a Person in a Scene](https://mlsanity.com/paper/2609.04603/) (2026)
- [RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures](https://mlsanity.com/paper/2609.39929/) (2026)
- [Soundwich: Video Generation with Layered and Controllable Audio](https://mlsanity.com/paper/2610.00691/) (2026)

## Abstract (arXiv)

Positional encodings are essential to transformer-based generative models, yet their behavior in multimodal and attention-sharing settings is not fully understood. In this work, we present a principled analysis of Rotary Positional Embeddings (RoPE), showing that RoPE naturally decomposes into frequency components with distinct positional sensitivities. We demonstrate that this frequency structure explains why shared-attention mechanisms, where a target image is generated while attending to tokens from a reference image, can lead to reference copying, in which the model reproduces content from the reference instead of extracting only its stylistic cues. Our analysis reveals that the high-frequency components of RoPE dominate the attention computation, forcing queries to attend mainly to spatially aligned reference tokens and thereby inducing this unintended copying behavior. Building on these insights, we introduce a method for selectively modulating RoPE frequency bands so that attention reflects semantic similarity rather than strict positional alignment. Applied to modern transformer-based diffusion architectures, where all tokens share attention, this modulation restores stable and meaningful shared attention. As a result, it enables effective control over the degree of style transfer versus content copying, yielding a proper style-aligned generation process in which stylistic attributes are transferred without duplicating reference content.