mlsanityData to 7 Oct 2026

Adaptive 3D-RoPE: Physics-Aligned Rotary Positional Encoding for Wireless Foundation Models

arXiv 2605.00968 ↗First posted 1 May 2026eess.SP cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Wireless foundation models (WFMs) have emerged as a promising paradigm for unified channel state information (CSI) acquisition across diverse tasks in sixth-generation (6G) networks. Although WFMs significantly outperform task-specific small models, their zero-shot cross-scenario generalization still remains limited for real-world applications. Existing positional embeddings, the sole interface through which self-attention perceives the temporal-frequency-antenna 3D physical coordinates of CSI, fail to capture the highly dynamic and axis-dependent coherence inherent in wireless channels. This paper proposes Adaptive 3D-RoPE, a channel-driven 3D rotary positional embedding framework for WFMs to dynamically align the 3D positional embeddings with the instantaneous coherence state of heterogeneous CSI. The design proceeds in three stages: first, an axis-wise learnable rotary prior independently preserves the temporal, frequency, and antenna coordinate structures; second, a feature-guided rotary modulation module maps the feature-wise standard deviation of visible CSI tokens to compact, sample-adaptive scales; third, identical coordinate offsets induce dynamically adjusted query-key interactions tailored to the instantaneous channel state. Extensive experiments on both simulated and measured datasets validate the effectiveness of Adaptive 3D-RoPE across three complementary dimensions. It reduces NMSE by 10.14, 6.25, and 4.61 dB relative to baselines under antenna, temporal, and frequency scaling, respectively. It transfers effectively to real-world measured CSI and remains robust under imperfect CSI. Finally, it transfers to the independently designed LWM backbone and beam-prediction task, improving zero-shot Top-1 accuracy by 8.03 percentage points.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

36 of 48 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  8. 2018Self-Attention with Relative Position Representationscited by 1,112
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  10. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  11. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  12. 2024Rotary Position Embedding for Vision Transformercited by 226
24 more
  1. 2022Sionna: An Open-Source Library for Next-Generation Physical Layer Researchcited by 143
  2. 2022A Length-Extrapolatable Transformercited by 123
  3. 2019DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applicationscited by 121
  4. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 111
  5. 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 63
  6. 2024LLM4CP: Adapting Large Language Models for Channel Predictioncited by 42
  7. 2024Large Wireless Model (LWM): A Foundation Model for Wireless Channelscited by 39
  8. 2025WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communicationcited by 39
  9. 2024WiFo: Wireless Foundation Model for Channel Predictioncited by 26
  10. 2024Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunitiescited by 25
  11. 2025Learning the RoPEs: Better 2D and 3D Position Encodings with STRINGcited by 19
  12. 2024HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolationcited by 17
  13. 2025VRoPE: Rotary Position Embedding for Video Large Language Modelscited by 17
  14. 2024LieRE: Lie Rotational Positional Encodingscited by 16
  15. 2025A MIMO Wireless Channel Foundation Model via CIR-CSI Consistencycited by 10
  16. 2025BERT4MIMO: A Foundation Model using BERT Architecture for Massive MIMO Channel State Information Predictioncited by 8
  17. 2025LVM4CSI: Enabling Direct Application of Pre-Trained Large Vision Models for Wireless Channel Taskscited by 7
  18. 2025WiFo-CF: Wireless Foundation Model for CSI Feedbackcited by 5
  19. 2026CSI-MAE: A Masked Autoencoder-based Channel Foundation Modelcited by 5
  20. 2026LWM-Temporal: Sparse Spatio-Temporal Attention for Wireless Channel Representation Learningcited by 5
  21. 2025Context-aware Rotary Position Embeddingcited by 4
  22. 2025WiFo-2: a generalist foundation model unifies heterogeneous wireless system designcited by 4
  23. 2026AI-Driven Channel State Information (CSI) Extrapolation for 6G: Current Situations, Challenges and Future Researchcited by 4
  24. 2026HeterCSI: Channel-Adaptive Heterogeneous CSI Pretraining Framework for Generalized Wireless Foundation Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.