# Complex Transformer: A Framework for Modeling Complex-Valued Sequence

arXiv: https://arxiv.org/abs/1910.10202 · HTML: https://mlsanity.com/paper/1910.10202/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 30 of 32)

- [Building Blocks for a Complex-Valued Transformer Architecture](https://mlsanity.com/paper/2306.09827/) (2023)
- [iFairy: the First 2-bit Complex LLM with All Parameters in ±1, ± i](https://mlsanity.com/paper/2508.05571/) (2025)
- [Expanding Expressivity in Transformer Models with M"obiusAttention](https://mlsanity.com/paper/2409.12175/) (2024)
- [Deep Sound Field Reconstruction in Real Rooms: Introducing the ISOBEL Sound Field Dataset](https://mlsanity.com/paper/2102.06455/) (2021)
- [Adversarial Audio Synthesis with Complex-valued Polynomial Networks](https://mlsanity.com/paper/2206.06811/) (2022)
- [CoNO: Complex Neural Operator for Continous Dynamical Physical Systems](https://mlsanity.com/paper/2406.02597/) (2024)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement](https://mlsanity.com/paper/2109.02011/) (2021)
- [Adapting Pre-trained Language Models for Quantum Natural Language Processing](https://mlsanity.com/paper/2302.13812/) (2023)
- [Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification](https://mlsanity.com/paper/2312.07338/) (2023)
- [BeatFormer: Efficient motion-robust remote heart rate estimation through unsupervised spectral zoomed attention filters](https://mlsanity.com/paper/2507.14885/) (2025)
- [Holographic Transformers for Complex-Valued Signal Processing: Integrating Phase Interference into Self-Attention](https://mlsanity.com/paper/2509.19331/) (2025)
- [Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in ± 1, ± i](https://mlsanity.com/paper/2512.02901/) (2025)
- [Fourier Transformers for Latent Crystallographic Diffusion and Generative Modeling](https://mlsanity.com/paper/2602.12045/) (2026)
- [The Quantum Shortcut: Complex Phase-State Dynamics Reduce the Optimization Steps of Sequence Models](https://mlsanity.com/paper/2608.14691/) (2026)
- [Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition](https://mlsanity.com/paper/2106.04392/) (2021)
- [Emerging Approaches for THz Array Imaging: A Tutorial Review and Software Tool](https://mlsanity.com/paper/2309.08844/) (2023)
- [Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adapter](https://mlsanity.com/paper/2309.09443/) (2023)
- [GMTR: Graph Matching Transformers](https://mlsanity.com/paper/2311.08141/) (2023)
- [DarkShot: Lighting Dark Images with Low-Compute and High-Quality](https://mlsanity.com/paper/2312.16805/) (2023)
- [TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processing](https://mlsanity.com/paper/2401.05431/) (2024)
- [TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networks](https://mlsanity.com/paper/2401.05432/) (2024)
- [DeepCSHAP: Utilizing Shapley Values to Explain Deep Complex-Valued Neural Networks](https://mlsanity.com/paper/2403.08428/) (2024)
- [T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Image](https://mlsanity.com/paper/2403.13663/) (2024)
- [Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithms](https://mlsanity.com/paper/2408.17101/) (2024)
- [Towards General Deepfake Detection with Dynamic Curriculum](https://mlsanity.com/paper/2410.11162/) (2024)
- [Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphs](https://mlsanity.com/paper/2410.13409/) (2024)
- [Mask6D: Masked Pose Priors For 6D Object Pose Estimation](https://mlsanity.com/paper/2507.06486/) (2025)
- [Unfolding Framework with Complex-Valued Deformable Attention for High-Quality Computer-Generated Hologram Generation](https://mlsanity.com/paper/2508.21657/) (2025)
- [Scaled Dot-Product Attention implements projection of inputs onto a common surface](https://mlsanity.com/paper/2602.02521/) (2026)

## Abstract (arXiv)

While deep learning has received a surge of interest in a variety of fields in recent years, major deep learning models barely use complex numbers. However, speech, signal and audio data are naturally complex-valued after Fourier Transform, and studies have shown a potentially richer representation of complex nets. In this paper, we propose a Complex Transformer, which incorporates the transformer model as a backbone for sequence modeling; we also develop attention and encoder-decoder network operating for complex input. The model achieves state-of-the-art performance on the MusicNet dataset and an In-phase Quadrature (IQ) signal dataset.