# The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA

arXiv: https://arxiv.org/abs/2609.03090 · HTML: https://mlsanity.com/paper/2609.03090/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 17 of its 19 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 17 of 17)

- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth](https://mlsanity.com/paper/2103.03404/) (2021)
- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [Implicit Regularization in Deep Learning May Not Be Explainable by Norms](https://mlsanity.com/paper/2005.06398/) (2020)
- [Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought](https://mlsanity.com/paper/2505.12514/) (2025)
- [LLM Latent Reasoning as Chain of Superposition](https://mlsanity.com/paper/2510.15522/) (2025)
- [Continuous Chain of Thought Enables Parallel Exploration and Reasoning](https://mlsanity.com/paper/2505.23648/) (2025)
- [Weight decay induces low-rank attention layers](https://mlsanity.com/paper/2410.23819/) (2024)
- [Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer](https://mlsanity.com/paper/2507.02199/) (2025)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [Addressing Token Uniformity in Transformers via Singular Value Transformation](https://mlsanity.com/paper/2208.11790/) (2022)
- [Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure](https://mlsanity.com/paper/2602.08783/) (2026)
- [The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models](https://mlsanity.com/paper/2604.06374/) (2026)
- [On State Reduction in Linear Attention](https://mlsanity.com/paper/2602.04852/) (2026)
- [State Rank Dynamics in Linear Attention LLMs](https://mlsanity.com/paper/2602.02195/) (2026)

## Built on by (most cited first, 2 of 2)

- [The Rank the Task Demands: A Causal Rank Law for Matrix Memories Trained on Group Composition](https://mlsanity.com/paper/2609.12259/) (2026)
- [When the Gradient Sees Rank: Provable Necessity, Causal Recruitment, and Composition in Trained Matrix Memories](https://mlsanity.com/paper/2609.17594/) (2026)

## Abstract (arXiv)

Continuous chain-of-thought models compress reasoning into latent tokens. Matrix-valued variants, which route each latent token through a d x d matrix bottleneck, introduce rank as a single-sample structural observable on the latent matrix Z. If matrix latents carry parallel reasoning paths via superposition, rank should track them, and truncating Z to low rank should hurt accuracy on tasks whose solutions plausibly require multiple components. Across four training regimes of a matrix-CODI model (three on ProsQA, one on GSM8K-Aug below the learning threshold), the rank-k projection ablation curve is flat to within 0.6 percentage points. A three-seed replication yields 81.0 +/- 2.0 percentage points accuracy while the final effective rank of Z spans {4, 12, 13}; the loss does not reward any particular rank. To test whether rank-blindness arises from the flatten-then-project readout alone, we trained four readouts: a bilinear reparametrization, a bilinear-plus-GELU readout nonlinear in Z, an SVD-augmented readout feeding singular values through an MLP, and a quadratic readout in Z Z^T. All four rank-k curves remain flat (Spearman p-values 0.63, 0.14, 0.82, 0.46). The flat curves persist for readouts nonlinear in Z. A linear probe on Z underperforms a raw pretrained hidden state at target prediction (AUC 0.673 vs. 0.846). A negative control on vanilla GPT-2 SFT (no matrix bottleneck, no Z, three seeds, n=500) reproduces a flat rank-k curve under the same intervention paradigm with pooled-mean range 0.20pp, and a random-h sensitivity floor lands at the same accuracy: the rank-k ablation alone conflates rank-blindness with position-irrelevance.