# Learning to Compare Longitudinal Images

arXiv: https://arxiv.org/abs/2304.02531 · HTML: https://mlsanity.com/paper/2304.02531/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [Deep Learning the City : Quantifying Urban Perception At A Global Scale](https://mlsanity.com/paper/1608.01769/) (2016)
- [Longitudinal Self-Supervised Learning](https://mlsanity.com/paper/2006.06930/) (2020)
- [F-CAM: Full Resolution Class Activation Maps via Guided Parametric Upscaling](https://mlsanity.com/paper/2109.07069/) (2021)
- [Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis](https://mlsanity.com/paper/2206.04281/) (2022)
- [Pairwise Learning to Rank by Neural Networks Revisited: Reconstruction, Theoretical Analysis and Practical Performance](https://mlsanity.com/paper/1909.02768/) (2019)
- [Learning distributions of shape trajectories from longitudinal datasets: a hierarchical model on a manifold of diffeomorphisms](https://mlsanity.com/paper/1803.10119/) (2018)
- [A hierarchical independent component analysis model for longitudinal Neuroimaging studies](https://mlsanity.com/paper/1808.01557/) (2018)

## Built on by (most cited first, 12 of 12)

- [A Framework for Interpretability in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2310.01685/) (2023)
- [Made to Order: Discovering monotonic temporal changes via self-supervised video ordering](https://mlsanity.com/paper/2404.16828/) (2024)
- [Longitudinal Self-supervised Learning Using Neural Ordinary Differential Equation](https://mlsanity.com/paper/2310.10431/) (2023)
- [Advancing Precision Oncology Through Modeling of Longitudinal and Multimodal Data](https://mlsanity.com/paper/2502.07836/) (2025)
- [Learning Temporally Equivariance for Degenerative Disease Progression in OCT by Predicting Future Representations](https://mlsanity.com/paper/2405.09404/) (2024)
- [A Progressive Risk Formulation for Enhanced Deep Learning based Total Knee Replacement Prediction in Knee Osteoarthritis](https://mlsanity.com/paper/2406.10119/) (2024)
- [V2C-Long: Longitudinal Cortex Reconstruction with Spatiotemporal Correspondence](https://mlsanity.com/paper/2402.17438/) (2024)
- [Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMs](https://mlsanity.com/paper/2509.24640/) (2025)
- [A Cautionary Tale of Self-Supervised Learning for Imaging Biomarkers: Alzheimer's Disease Case Study](https://mlsanity.com/paper/2601.16467/) (2026)
- [AI-Based Detection of Temporal Changes in MR-Linac Images Acquired During Routine Prostate Radiotherapy](https://mlsanity.com/paper/2602.04983/) (2026)
- [Chronological Contrastive Learning: Few-Shot Progression Assessment in Irreversible Diseases](https://mlsanity.com/paper/2603.21935/) (2026)
- [Data-driven techniques for translational neuroscience and personalized neuro-health](https://mlsanity.com/paper/2608.13749/) (2026)

## Abstract (arXiv)

Longitudinal studies, where a series of images from the same set of individuals are acquired at different time-points, represent a popular technique for studying and characterizing temporal dynamics in biomedical applications. The classical approach for longitudinal comparison involves normalizing for nuisance variations, such as image orientation or contrast differences, via pre-processing. Statistical analysis is, in turn, conducted to detect changes of interest, either at the individual or population level. This classical approach can suffer from pre-processing issues and limitations of the statistical modeling. For example, normalizing for nuisance variation might be hard in settings where there are a lot of idiosyncratic changes. In this paper, we present a simple machine learning-based approach that can alleviate these issues. In our approach, we train a deep learning model (called PaIRNet, for Pairwise Image Ranking Network) to compare pairs of longitudinal images, with or without supervision. In the self-supervised setup, for instance, the model is trained to temporally order the images, which requires learning to recognize time-irreversible changes. Our results from four datasets demonstrate that PaIRNet can be very effective in localizing and quantifying meaningful longitudinal changes while discounting nuisance variation. Our code is available at urlhttps://github.com/heejong-kim/learning-to-compare-longitudinal-images.git