# Adaptive 3D-RoPE: Physics-Aligned Rotary Positional Encoding for Wireless Foundation Models

arXiv: https://arxiv.org/abs/2605.00968 · HTML: https://mlsanity.com/paper/2605.00968/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 48 references that are in the map.
- First posted on arXiv 1 May 2026.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens](https://mlsanity.com/paper/2402.13753/) (2024)
- [Rotary Position Embedding for Vision Transformer](https://mlsanity.com/paper/2403.13298/) (2024)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [A Length-Extrapolatable Transformer](https://mlsanity.com/paper/2212.10554/) (2022)
- [DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applications](https://mlsanity.com/paper/1902.06435/) (2019)
- [Round and Round We Go! What makes Rotary Positional Encodings useful?](https://mlsanity.com/paper/2410.06205/) (2024)
- [VideoRoPE: What Makes for Good Video Rotary Position Embedding?](https://mlsanity.com/paper/2502.05173/) (2025)
- [LLM4CP: Adapting Large Language Models for Channel Prediction](https://mlsanity.com/paper/2406.14440/) (2024)
- [Large Wireless Model (LWM): A Foundation Model for Wireless Channels](https://mlsanity.com/paper/2411.08872/) (2024)
- [WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication](https://mlsanity.com/paper/2502.06877/) (2025)
- [WiFo: Wireless Foundation Model for Channel Prediction](https://mlsanity.com/paper/2412.08908/) (2024)
- [Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunities](https://mlsanity.com/paper/2412.14538/) (2024)
- [Learning the RoPEs: Better 2D and 3D Position Encodings with STRING](https://mlsanity.com/paper/2502.02562/) (2025)
- [LieRE: Lie Rotational Positional Encodings](https://mlsanity.com/paper/2406.10322/) (2024)
- [VRoPE: Rotary Position Embedding for Video Large Language Models](https://mlsanity.com/paper/2502.11664/) (2025)
- [HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation](https://mlsanity.com/paper/2410.21216/) (2024)
- [A MIMO Wireless Channel Foundation Model via CIR-CSI Consistency](https://mlsanity.com/paper/2502.11965/) (2025)
- [BERT4MIMO: A Foundation Model using BERT Architecture for Massive MIMO Channel State Information Prediction](https://mlsanity.com/paper/2501.01802/) (2025)
- [LVM4CSI: Enabling Direct Application of Pre-Trained Large Vision Models for Wireless Channel Tasks](https://mlsanity.com/paper/2507.05121/) (2025)
- [CSI-MAE: A Masked Autoencoder-based Channel Foundation Model](https://mlsanity.com/paper/2601.03789/) (2026)

## Built on by (most cited first, 2 of 2)

- [Towards CSI-Native Foundation Models: A Channel-Adaptive Roadmap for 6G](https://mlsanity.com/paper/2606.20670/) (2026)
- [A Comprehensive Survey of Wireless Foundation Models for AI-Native 6G Networks](https://mlsanity.com/paper/2608.14694/) (2026)

## Abstract (arXiv)

Wireless foundation models (WFMs) have emerged as a promising paradigm for unified channel state information (CSI) acquisition across diverse tasks in sixth-generation (6G) networks. Although WFMs significantly outperform task-specific small models, their zero-shot cross-scenario generalization still remains limited for real-world applications. Existing positional embeddings, the sole interface through which self-attention perceives the temporal-frequency-antenna 3D physical coordinates of CSI, fail to capture the highly dynamic and axis-dependent coherence inherent in wireless channels. This paper proposes Adaptive 3D-RoPE, a channel-driven 3D rotary positional embedding framework for WFMs to dynamically align the 3D positional embeddings with the instantaneous coherence state of heterogeneous CSI. The design proceeds in three stages: first, an axis-wise learnable rotary prior independently preserves the temporal, frequency, and antenna coordinate structures; second, a feature-guided rotary modulation module maps the feature-wise standard deviation of visible CSI tokens to compact, sample-adaptive scales; third, identical coordinate offsets induce dynamically adjusted query-key interactions tailored to the instantaneous channel state. Extensive experiments on both simulated and measured datasets validate the effectiveness of Adaptive 3D-RoPE across three complementary dimensions. It reduces NMSE by 10.14, 6.25, and 4.61 dB relative to baselines under antenna, temporal, and frequency scaling, respectively. It transfers effectively to real-world measured CSI and remains robust under imperfect CSI. Finally, it transfers to the independently designed LWM backbone and beam-prediction task, improving zero-shot Top-1 accuracy by 8.03 percentage points.