Generic Triple-Latent Compression with Gated Associative Retrieval
Not on the map. Why?
Abstract (arXiv)
We study generic triple-latent sequence models that maintain a running token state and compressed pair-memory pathway to capture higher-order token interactions without benchmark-specific parsing. The triple-latent family improves a small Transformer baseline on byte-level WikiText-2 and on a tokenizer-based MiniMind language-model benchmark, while a recall-focused gated key-value retrieval extension improves associative recall but remains seed-sensitive and much slower in the current reference implementation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
12 of 16 references.
References · 12
- 2017Attention Is All You Needcited by 59,932
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2018Universal Transformerscited by 613
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
- 2022Memorizing Transformerscited by 219
- 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 167
- 2019Large Memory Layers with Product Keyscited by 120
Built on by
Not cited yet.