# Deriving Neural Architectures from Sequence and Graph Kernels

arXiv: https://arxiv.org/abs/1705.09037 · HTML: https://mlsanity.com/paper/1705.09037/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 23 of 23)

- [Geometric Deep Learning on Molecular Representations](https://mlsanity.com/paper/2107.12375/) (2021)
- [Multi-Scale Representation Learning on Proteins](https://mlsanity.com/paper/2204.02337/) (2022)
- [KerGNNs: Interpretable Graph Neural Networks with Graph Kernels](https://mlsanity.com/paper/2201.00491/) (2022)
- [Theoretically Improving Graph Neural Networks via Anonymous Walk Graph Kernels](https://mlsanity.com/paper/2104.02995/) (2021)
- [Hashing-Accelerated Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2105.14280/) (2021)
- [Graph Kernel Neural Networks](https://mlsanity.com/paper/2112.07436/) (2021)
- [ReactionT5: a large-scale pre-trained model towards application of limited reaction data](https://mlsanity.com/paper/2311.06708/) (2023)
- [Weisfeiler and Leman Go Walking: Random Walk Kernels Revisited](https://mlsanity.com/paper/2205.10914/) (2022)
- [Ensemble Spectral Prediction (ESP) Model for Metabolite Annotation](https://mlsanity.com/paper/2203.13783/) (2022)
- [State of the Art and Potentialities of Graph-level Learning](https://mlsanity.com/paper/2301.05860/) (2023)
- [Descriptive Kernel Convolution Network with Improved Random Walk Kernel](https://mlsanity.com/paper/2402.06087/) (2024)
- [Computing Steiner Trees using Graph Neural Networks](https://mlsanity.com/paper/2108.08368/) (2021)
- [Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification](https://mlsanity.com/paper/2301.13764/) (2023)
- [ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions](https://mlsanity.com/paper/2111.06222/) (2021)
- [Self-supervision meets kernel graph neural models: From architecture to augmentations](https://mlsanity.com/paper/2310.11281/) (2023)
- [Graph-level Neural Networks: Current Progress and Future Directions](https://mlsanity.com/paper/2205.15555/) (2022)
- [VacciNet: Towards a Smart Framework for Learning the Distribution Chain Optimization of Vaccines for a Pandemic](https://mlsanity.com/paper/2208.01112/) (2022)
- [Nearly Optimal Steiner Trees using Graph Neural Network Assisted Monte Carlo Tree Search](https://mlsanity.com/paper/2305.00535/) (2023)
- [Towards Subgraph Isomorphism Counting with Graph Kernels](https://mlsanity.com/paper/2405.07497/) (2024)
- [Beyond 5G Network Failure Classification for Network Digital Twin Using Graph Neural Network](https://mlsanity.com/paper/2406.06595/) (2024)
- [log-RRIM: Yield Prediction via Local-to-global Reaction Representation Learning and Interaction Modeling](https://mlsanity.com/paper/2411.03320/) (2024)
- [Graph Neural Network-based Spectral Filtering Mechanism for Imbalance Classification in Network Digital Twin](https://mlsanity.com/paper/2502.11505/) (2025)
- [Graph Lineages and Skeletal Graph Products](https://mlsanity.com/paper/2508.00197/) (2025)

## Abstract (arXiv)

The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process. In this work, we appeal to kernels over combinatorial structures, such as sequences and graphs, to derive appropriate neural operations. We introduce a class of deep recurrent neural operations and formally characterize their associated kernel spaces. Our recurrent modules compare the input to virtual reference objects (cf. filters in CNN) via the kernels. Similar to traditional neural operations, these reference objects are parameterized and directly optimized in end-to-end training. We empirically evaluate the proposed class of neural architectures on standard applications such as language modeling and molecular graph regression, achieving state-of-the-art results across these applications.