# Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling

arXiv: https://arxiv.org/abs/2312.03523 · HTML: https://mlsanity.com/paper/2312.03523/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [All-but-the-Top: Simple and Effective Postprocessing for Word Representations](https://mlsanity.com/paper/1702.01417/) (2017)
- [RumourEval 2019: Determining Rumour Veracity and Support for Rumours](https://mlsanity.com/paper/1809.06683/) (2018)
- [Deep Signature Transforms](https://mlsanity.com/paper/1905.08494/) (2019)
- [Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads](https://mlsanity.com/paper/1511.07487/) (2015)
- [All-in-one: Multi-task Learning for Rumour Verification](https://mlsanity.com/paper/1806.03713/) (2018)
- [Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding](https://mlsanity.com/paper/2203.01515/) (2022)
- [Dynamic Word Embeddings](https://mlsanity.com/paper/1702.08359/) (2017)
- [Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU](https://mlsanity.com/paper/2001.00706/) (2020)
- [The Dynamic Embedded Topic Model](https://mlsanity.com/paper/1907.05545/) (2019)
- [The iisignature library: efficient calculation of iterated-integral signatures and log signatures](https://mlsanity.com/paper/1802.08252/) (2018)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Dynamic Word Embeddings for Evolving Semantic Discovery](https://mlsanity.com/paper/1703.00607/) (2017)
- [Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention](https://mlsanity.com/paper/1910.12038/) (2019)
- [Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection](https://mlsanity.com/paper/1704.05973/) (2017)
- [Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition](https://mlsanity.com/paper/1707.03993/) (2017)
- [Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition](https://mlsanity.com/paper/1610.02616/) (2016)
- [Speaker Turn Modeling for Dialogue Act Classification](https://mlsanity.com/paper/2109.05056/) (2021)
- [Identifying Moments of Change from Longitudinal User Text](https://mlsanity.com/paper/2205.05593/) (2022)
- [A signature-based machine learning model for bipolar disorder and borderline personality disorder](https://mlsanity.com/paper/1707.07124/) (2017)
- [Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning](https://mlsanity.com/paper/2205.14794/) (2022)
- [Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition](https://mlsanity.com/paper/2110.13008/) (2021)
- [Modelling Temporal Document Sequences for Clinical ICD Coding](https://mlsanity.com/paper/2302.12666/) (2023)
- [Information Extraction from Swedish Medical Prescriptions with Sig-Transformer Encoder](https://mlsanity.com/paper/2010.04897/) (2020)

## Built on by (most cited first, 2 of 2)

- [TempoFormer: A Transformer for Temporally-aware Representations in Change Detection](https://mlsanity.com/paper/2408.15689/) (2024)
- [LiFT: How to Enable In-Context Learning for Longitudinal Modelling](https://mlsanity.com/paper/2604.16382/) (2026)

## Abstract (arXiv)

We present an open-source, pip installable toolkit, Sig-Networks, the first of its kind for longitudinal language modelling. A central focus is the incorporation of Signature-based Neural Network models, which have recently shown success in temporal tasks. We apply and extend published research providing a full suite of signature-based models. Their components can be used as PyTorch building blocks in future architectures. Sig-Networks enables task-agnostic dataset plug-in, seamless pre-processing for sequential data, parameter flexibility, automated tuning across a range of models. We examine signature networks under three different NLP tasks of varying temporal granularity: counselling conversations, rumour stance switch and mood changes in social media threads, showing SOTA performance in all three, and provide guidance for future tasks. We release the Toolkit as a PyTorch package with an introductory video, Git repositories for preprocessing and modelling including sample notebooks on the modeled NLP tasks.