# An Algorithm for Routing Vectors in Sequences

arXiv: https://arxiv.org/abs/2211.11754 · HTML: https://mlsanity.com/paper/2211.11754/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [DeepCaps: Going Deeper with Capsule Networks](https://mlsanity.com/paper/1904.09546/) (2019)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Capsules with Inverted Dot-Product Attention Routing](https://mlsanity.com/paper/2002.04764/) (2020)
- [Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement](https://mlsanity.com/paper/1902.05770/) (2019)
- [Fast Dynamic Routing Based on Weighted Kernel Density Estimation](https://mlsanity.com/paper/1805.10807/) (2018)
- [An Algorithm for Routing Capsules in All Domains](https://mlsanity.com/paper/1911.00792/) (2019)

## Built on by (most cited first, 7 of 7)

- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [Informed Named Entity Recognition Decoding for Generative Language Models](https://mlsanity.com/paper/2308.07791/) (2023)
- [LlamBERT: Large-scale low-cost data annotation in NLP](https://mlsanity.com/paper/2403.15938/) (2024)
- [ElegansNet: a brief scientific report and initial experiments](https://mlsanity.com/paper/2304.13538/) (2023)
- [Re-Representation in Sentential Relation Extraction with Sequence Routing Algorithm](https://mlsanity.com/paper/2508.21049/) (2025)
- [Multi-refined Feature Enhanced Sentiment Analysis Using Contextual Instruction](https://mlsanity.com/paper/2511.00537/) (2025)
- [The Association of Transformer-based Sentiment Analysis with Symptom Distress and Deterioration in Routine Psychotherapy Care](https://mlsanity.com/paper/2605.09838/) (2026)

## Abstract (arXiv)

We propose a routing algorithm that takes a sequence of vectors and computes a new sequence with specified length and vector size. Each output vector maximizes "bang per bit," the difference between a net benefit to use and net cost to ignore data, by better predicting the input vectors. We describe output vectors as geometric objects, as latent variables that assign credit, as query states in a model of associative memory, and as agents in a model of a Society of Mind. We implement the algorithm with optimizations that reduce parameter count, computation, and memory use by orders of magnitude, enabling us to route sequences of greater length than previously possible. We evaluate our implementation on natural language and visual classification tasks, obtaining competitive or state-of-the-art accuracy and end-to-end credit assignments that are interpretable.