mlsanityData to 14 Oct 2026

Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks

arXiv 1906.04393 ↗First posted 11 Jun 2019cs.CL cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Many state-of-the-art neural models for NLP are heavily parameterized and thus memory inefficient. This paper proposes a series of lightweight and memory efficient neural architectures for a potpourri of natural language processing (NLP) tasks. To this end, our models exploit computation using Quaternion algebra and hypercomplex spaces, enabling not only expressive inter-component interactions but also significantly (75%) reduced parameter size due to lesser degrees of freedom in the Hamilton product. We propose Quaternion variants of models, giving rise to new architectures such as the Quaternion attention Model and Quaternion Transformer. Extensive experiments on a battery of NLP tasks demonstrates the utility of proposed Quaternion-inspired models, enabling up to 75% reduction in parameter size without significant loss in performance.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 2; 2023: 3; 2024: 3; 2025: 4; 2026: 4.

26 papers.

Sort:

By area:

  1. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
  2. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 106
  3. 2021Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameterscited by 66
  4. 2021PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutionscited by 26
  5. 2021Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddingscited by 12
  6. 2023Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentationcited by 10
  7. 2021Parameterized Hypercomplex Graph Neural Networks for Graph Classificationcited by 7
  8. 2022Learning Speech Emotion Representations in the Quaternion Domaincited by 7
  9. 2022Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attentioncited by 6
  10. 2021Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modellingcited by 4
  11. 2025DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Accelerationcited by 4
  12. 2021Two-view Graph Neural Networks for Knowledge Graph Completioncited by 3
14 more
  1. 2025AI Progress Should Be Measured by Capability-Per-Resource, Not Scale Alone: A Framework for Gradient-Guided Resource Allocation in LLMscited by 3
  2. 2023Adapting Pre-trained Language Models for Quantum Natural Language Processingcited by 2
  3. 2025Deconstructing Attention: Investigating Design Principles for Effective Language Modelingcited by 2
  4. 2024Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learningcited by 1
  5. 2024Improving Quaternion Neural Networks with Quaternionic Activation Functionscited by 1
  6. 2026Quaternion Self-Attention with Shared Scorescited by 1
  7. 2021Efficient Attribute Injection for Pretrained Language Modelscited by 0
  8. 2021Quaternion-Based Graph Convolution Network for Recommendationcited by 0
  9. 2023Experimental Results regarding multiple Machine Learning via Quaternionscited by 0
  10. 2024An Effective, Robust and Fairness-aware Hate Speech Detection Frameworkcited by 0
  11. 2025Downsized and Compromised?: Assessing the Faithfulness of Model Compressioncited by 0
  12. 2026QC-GAN: A Parameter-Efficient Quaternion Conformer GAN for High-Fidelity Speech Enhancementcited by 0
  13. 2026Bridge the Gaps: Heterogeneous Attributed Graph Clustering via Quaternion Representation Learningcited by 0
  14. 2026QSV: Quat-Sphere-Vision for Coupled Quaternion Attention on Spherical Latticescited by 0