mlsanityData to 14 Oct 2026

LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization

arXiv 2505.05893 ↗First posted 9 May 2025cs.AR cs.AI cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in Protein Structure Prediction Models (PPMs), such as AlphaFold2 and ESMFold, have revolutionized computational biology by achieving unprecedented accuracy in predicting three-dimensional protein folding structures. However, these models face significant scalability challenges, particularly when processing proteins with long amino acid sequences (e.g., sequence length > 1,000). The primary bottleneck that arises from the exponential growth in activation sizes is driven by the unique data structure in PPM, which introduces an additional dimension that leads to substantial memory and computational demands. These limitations have hindered the effective scaling of PPM for real-world applications, such as analyzing large proteins or complex multimers with critical biological and pharmaceutical relevance. In this paper, we present LightNobel, the first hardware-software co-designed accelerator developed to overcome scalability limitations on the sequence length in PPM. At the software level, we propose Token-wise Adaptive Activation Quantization (AAQ), which leverages unique token-wise characteristics, such as distogram patterns in PPM activations, to enable fine-grained quantization techniques without compromising accuracy. At the hardware level, LightNobel integrates the multi-precision reconfigurable matrix processing unit (RMPU) and versatile vector processing unit (VVPU) to enable the efficient execution of AAQ. Through these innovations, LightNobel achieves up to 8.44x, 8.41x speedup and 37.29x, 43.35x higher power efficiency over the latest NVIDIA A100 and H100 GPUs, respectively, while maintaining negligible accuracy loss. It also reduces the peak memory requirement up to 120.05x in PPM, enabling scalable processing for proteins with long sequences.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 58 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

17 of 58 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  6. 2015An Introduction to Convolutional Neural Networkscited by 514
  7. 2021A White Paper on Neural Network Quantizationcited by 472
  8. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  9. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  10. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  11. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 106
  12. 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 101
5 more
  1. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  2. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  3. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 28
  4. 2022Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Modelscited by 19
  5. 2024ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hourscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models
  2. 2025Scaling Up ESM2 Architectures for Long Protein Sequences Analysis: Long and Quantized Approaches
  3. 2022AlphaFold Distillation for Protein Design
  4. 2025Protein Structure Tokenization: Benchmarking and New Recipe
  5. 2023Exploring Post-Training Quantization of Protein Language Models
  6. 2024FoldToken2: Learning compact, invariant and generative protein structure language
  7. 2023Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation
  8. 2025Protenix-Mini+: efficient structure prediction model with scalable pairformer