# Recurrent Kernel Networks

arXiv: https://arxiv.org/abs/1906.03200 · HTML: https://mlsanity.com/paper/1906.03200/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [Convolutional Deep Kernel Machines](https://mlsanity.com/paper/2309.09814/) (2023)
- [Convolutional Motif Kernel Networks](https://mlsanity.com/paper/2111.02272/) (2021)
- [CECILIA: Comprehensive Secure Machine Learning Framework](https://mlsanity.com/paper/2202.03023/) (2022)
- [COmic: Convolutional Kernel Networks for Interpretable End-to-End Learning on (Multi-)Omics Data](https://mlsanity.com/paper/2212.02504/) (2022)
- [On Inductive Biases for Machine Learning in Data Constrained Settings](https://mlsanity.com/paper/2302.10692/) (2023)

## Abstract (arXiv)

Substring kernels are classical tools for representing biological sequences or text. However, when large amounts of annotated data are available, models that allow end-to-end training such as neural networks are often preferred. Links between recurrent neural networks (RNNs) and substring kernels have recently been drawn, by formally showing that RNNs with specific activation functions were points in a reproducing kernel Hilbert space (RKHS). In this paper, we revisit this link by generalizing convolutional kernel networks---originally related to a relaxation of the mismatch kernel---to model gaps in sequences. It results in a new type of recurrent neural network which can be trained end-to-end with backpropagation, or without supervision by using kernel approximation techniques. We experimentally show that our approach is well suited to biological sequences, where it outperforms existing methods for protein classification tasks.