# Self-Supervised Longitudinal Neighbourhood Embedding

arXiv: https://arxiv.org/abs/2103.03840 · HTML: https://mlsanity.com/paper/2103.03840/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Diffusion Improves Graph Learning](https://mlsanity.com/paper/1911.05485/) (2019)
- [Revisiting Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1901.09005/) (2019)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Longitudinal Self-Supervised Learning](https://mlsanity.com/paper/2006.06930/) (2020)
- [Non-Rigid Image Registration Using Self-Supervised Fully Convolutional Networks without Training Data](https://mlsanity.com/paper/1801.04012/) (2018)
- [Unsupervised Deep Learning for Bayesian Brain MRI Segmentation](https://mlsanity.com/paper/1904.11319/) (2019)
- [Diagnosis of Alzheimer's Disease via Multi-modality 3D Convolutional Neural Network](https://mlsanity.com/paper/1902.09904/) (2019)
- [Self-Supervised Representation Learning via Neighborhood-Relational Encoding](https://mlsanity.com/paper/1908.10455/) (2019)
- [Longitudinal Pooling & Consistency Regularization to Model Disease Progression from MRIs](https://mlsanity.com/paper/2003.13958/) (2020)

## Built on by (most cited first, 16 of 16)

- [Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis](https://mlsanity.com/paper/2206.04281/) (2022)
- [Generative Aging of Brain Images with Diffeomorphic Registration](https://mlsanity.com/paper/2205.15607/) (2022)
- [LMT: Longitudinal Mixing Training, a Framework to Predict Disease Progression from a Single Image](https://mlsanity.com/paper/2310.10420/) (2023)
- [Longitudinal Self-supervised Learning Using Neural Ordinary Differential Equation](https://mlsanity.com/paper/2310.10431/) (2023)
- [Advancing Precision Oncology Through Modeling of Longitudinal and Multimodal Data](https://mlsanity.com/paper/2502.07836/) (2025)
- [Dual Meta-Learning with Longitudinally Generalized Regularization for One-Shot Brain Tissue Segmentation Across the Human Lifespan](https://mlsanity.com/paper/2308.06774/) (2023)
- [LSOR: Longitudinally-Consistent Self-Organized Representation Learning](https://mlsanity.com/paper/2310.00213/) (2023)
- [Spatiotemporal Representation Learning for Short and Long Medical Image Time Series](https://mlsanity.com/paper/2403.07513/) (2024)
- [Detection of diabetic retinopathy using longitudinal self-supervised learning](https://mlsanity.com/paper/2209.00915/) (2022)
- [Identity Preserving Latent Diffusion for Brain Aging Modeling](https://mlsanity.com/paper/2503.09634/) (2025)
- [Data-Efficient Fine-Tuning of Vision-Language Models for Diagnosis of Alzheimer's Disease](https://mlsanity.com/paper/2509.07613/) (2025)
- [Parcel2Progression: An Anatomy-aware Longitudinal Framework for Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2608.08753/) (2026)
- [Impact of Initialization on Intra-subject Pediatric Brain MR Image Registration: A Comparative Analysis between SyN ANTs and Deep Learning-Based Approaches](https://mlsanity.com/paper/2406.19943/) (2024)
- [Self-Supervised Cross-Encoder for Neurodegenerative Disease Diagnosis](https://mlsanity.com/paper/2509.07623/) (2025)
- [Semi-disentangled spatiotemporal implicit neural representations of longitudinal neuroimaging data for trajectory classification](https://mlsanity.com/paper/2510.09936/) (2025)
- [Longitudinal Bayesian Learning of Continuous Disease Position across the Alzheimer's Disease Continuum](https://mlsanity.com/paper/2608.19436/) (2026)

## Abstract (arXiv)

Longitudinal MRIs are often used to capture the gradual deterioration of brain structure and function caused by aging or neurological diseases. Analyzing this data via machine learning generally requires a large number of ground-truth labels, which are often missing or expensive to obtain. Reducing the need for labels, we propose a self-supervised strategy for representation learning named Longitudinal Neighborhood Embedding (LNE). Motivated by concepts in contrastive learning, LNE explicitly models the similarity between trajectory vectors across different subjects. We do so by building a graph in each training iteration defining neighborhoods in the latent space so that the progression direction of a subject follows the direction of its neighbors. This results in a smooth trajectory field that captures the global morphological change of the brain while maintaining the local continuity. We apply LNE to longitudinal T1w MRIs of two neuroimaging studies: a dataset composed of 274 healthy subjects, and Alzheimer's Disease Neuroimaging Initiative (ADNI, N=632). The visualization of the smooth trajectory vector field and superior performance on downstream tasks demonstrate the strength of the proposed method over existing self-supervised methods in extracting information associated with normal aging and in revealing the impact of neurodegenerative disorders. The code is available at urlhttps://github.com/ouyangjiahong/longitudinal-neighbourhood-embedding.git.