# CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering

arXiv: https://arxiv.org/abs/2610.09476 · HTML: https://mlsanity.com/paper/2610.09476/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Efficient LLM inference and serving

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Fast Transformer Decoding: One Write-Head is All You Need](https://mlsanity.com/paper/1911.02150/) (2019)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://mlsanity.com/paper/2403.03853/) (2024)
- [Qwen2.5-1M Technical Report](https://mlsanity.com/paper/2501.15383/) (2025)
- [Optimised Grouped-Query Attention Mechanism for Transformers](https://mlsanity.com/paper/2406.14963/) (2024)
- [RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Models](https://mlsanity.com/paper/2510.07782/) (2025)
- [Geometric and Spectral Alignment for Deep Neural Network II](https://mlsanity.com/paper/2605.02111/) (2026)
- [Geometric and Spectral Alignment for Deep Neural Network I](https://mlsanity.com/paper/2605.02108/) (2026)
- [CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation](https://mlsanity.com/paper/2607.02576/) (2026)

## Abstract (arXiv)

Geometric and Spectral Alignment (GSA) characterizes trained networks through spectral concentration, physical-channel alignment, support structure, and changes in singular bases. In this paper, we propose CHASE (Channel-Aligned Structure Exploitation) to use these structures in practical model design. CHASE covers six applications across model modification, reconfiguration, and compression. CORA, COEC, and CORAM apply GSA to parameter-efficient finetuning, structured-pruning compensation, and model merging. We further develop three new methods. CAGA uses GSA to identify multi-head attention heads that can share a KV representation and constructs the shared key and value heads through geometric alignment and low-rank subspace extraction. SAKV uses GSA to determine which adjacent layers can share a low-rank KV-cache representation and the retained rank for each layer group. CAPS uses GSA spectral structure to group output neurons and selects retained input channels separately for each group. Results from CORA, COEC, and CORAM establish the effectiveness of GSA for adaptation, pruning compensation, and model merging. Experiments on CAGA show that geometric shared-head construction substantially improves MHA-to-GQA conversion, and SAKV and CAPS improve over representative baselines for KV-cache compression and structured pruning. These results show that the structures identified by GSA can be used directly to design methods for a range of model operations.