# Flash Invariant Point Attention

arXiv: https://arxiv.org/abs/2505.11580 · HTML: https://mlsanity.com/paper/2505.11580/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 16 of 16)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [SE(3) diffusion model with application to protein backbone generation](https://mlsanity.com/paper/2302.02277/) (2023)
- [SE(3)-Stochastic Flow Matching for Protein Backbone Generation](https://mlsanity.com/paper/2310.02391/) (2023)
- [Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds](https://mlsanity.com/paper/2301.12485/) (2023)
- [Improved motif-scaffolding with SE(3) flow matching](https://mlsanity.com/paper/2401.04082/) (2024)
- [Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation](https://mlsanity.com/paper/2405.20313/) (2024)
- [gRNAde: Geometric Deep Learning for 3D RNA inverse design](https://mlsanity.com/paper/2305.14749/) (2023)
- [Accurate RNA 3D structure prediction using a language model-based deep learning approach](https://mlsanity.com/paper/2207.01586/) (2022)
- [Generating Highly Designable Proteins with Geometric Algebra Flow Matching](https://mlsanity.com/paper/2411.05238/) (2024)
- [P2DFlow: A Protein Ensemble Generative Model with SE(3) Flow Matching](https://mlsanity.com/paper/2411.17196/) (2024)
- [De novo antibody design with SE(3) diffusion](https://mlsanity.com/paper/2405.07622/) (2024)
- [RNA-FrameFlow: Flow Matching for de novo 3D RNA Backbone Design](https://mlsanity.com/paper/2406.13839/) (2024)
- [Floating Anchor Diffusion Model for Multi-motif Scaffolding](https://mlsanity.com/paper/2406.03141/) (2024)

## Built on by (most cited first, 1 of 1)

- [Design-CP: Context Parallelism for Design of Protein Nanoparticles](https://mlsanity.com/paper/2607.05439/) (2026)

## Abstract (arXiv)

Invariant Point Attention (IPA) is a key algorithm for geometry-aware modeling in structural biology, central to many protein and RNA models. However, its quadratic complexity limits the input sequence length. We introduce FlashIPA, a factorized reformulation of IPA that leverages hardware-efficient FlashAttention to achieve linear scaling in GPU memory and wall-clock time with sequence length. FlashIPA matches or exceeds standard IPA performance while substantially reducing computational costs. FlashIPA extends training to previously unattainable lengths, and we demonstrate this by re-training generative models without length restrictions and generating structures of thousands of residues. FlashIPA is available at https://github.com/flagshippioneering/flash_ipa.