# Same path, different: a mechanistic comparison of looped and stacked transformer encoders on 12-lead ECG

arXiv: https://arxiv.org/abs/2609.15498 · HTML: https://mlsanity.com/paper/2609.15498/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL](https://mlsanity.com/paper/2004.13701/) (2020)
- [MiniViT: Compressing Vision Transformers with Weight Multiplexing](https://mlsanity.com/paper/2204.07154/) (2022)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [Sliced Recursive Transformer](https://mlsanity.com/paper/2111.05297/) (2021)
- [Investigating Single-Block Recurrence in Vision Transformers for Image Recognition](https://mlsanity.com/paper/2605.10661/) (2026)
- [Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision](https://mlsanity.com/paper/2608.04879/) (2026)

## Built on by (most cited first, 1 of 1)

- [Monitoring latent reasoning in looped language models](https://mlsanity.com/paper/2610.18127/) (2026)

## Abstract (arXiv)

Recurrent Transformers reusing their weights rather than stacking L distinct layers are becoming widely adopted due to their parameter efficiency [1,2,3]. However, the exact representational and dynamical differences between looped and stacked architectures remain uncharacterized. This paper presents a controlled study on the example of bViT model [1] applying one weight-tied block L times. We train two models: bViT and standard ViT [4] on 12-lead electrocardiogram (ECG) classification tasks from the PTB-XL dataset under identical training protocols. Despite an 8.9× parameter reduction, bViT achieves accuracy parity with ViT. Geometric similarity metrics demonstrate that both architectures construct comparable latent representations in an equivalent canonical order. Crucially, their dynamics differ: bViT exhibits smaller step sizes and inter-patient sensitivity, as well as near-neutral behavior away from the data manifold, whereas ViT exhibits collapsing dimensionality of representations and out-of-distribution feature expansion.