# Deep Shape Regression for Planar Curves with Multimodal Covariates

arXiv: https://arxiv.org/abs/2607.19600 · HTML: https://mlsanity.com/paper/2607.19600/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Automated brain extraction of multi-sequence MRI using artificial neural networks](https://mlsanity.com/paper/1901.11341/) (2019)
- [Deep Learning for Functional Data Analysis with Adaptive Basis Layers](https://mlsanity.com/paper/2106.10414/) (2021)
- [Non-linear Functional Modeling using Neural Networks](https://mlsanity.com/paper/2104.09371/) (2021)
- [Elastic analysis of irregularly or sparsely sampled curves](https://mlsanity.com/paper/2104.11039/) (2021)
- [CovNet: Covariance Networks for Functional Data on Multidimensional Domains](https://mlsanity.com/paper/2104.05021/) (2021)
- [Neural Networks for Scalar Input and Functional Output](https://mlsanity.com/paper/2208.05776/) (2022)
- [Fast symmetric additive covariance smoothing](https://mlsanity.com/paper/1609.07007/) (2016)
- [Functional additive models on manifolds of planar shapes and forms](https://mlsanity.com/paper/2109.02624/) (2021)
- [Nonlinear Functional Principal Component Analysis Using Neural Networks](https://mlsanity.com/paper/2306.14388/) (2023)
- [Functional PCA with Covariate Dependent Mean and Covariance Structure](https://mlsanity.com/paper/2001.11425/) (2020)
- [Elastic Full Procrustes Analysis of Plane Curves via Hermitian Covariance Smoothing](https://mlsanity.com/paper/2203.10522/) (2022)

## Abstract (arXiv)

The shape of a planar curve is the geometric information that remains once translation, rotation, scale and reparametrisation are removed and is of interest in many health applications, e.g. in neuroimaging. We propose a deep shape regression model for open planar curves that admits multimodal and high-dimensional covariates. Representing curves as complex-valued functions, we show that the conditional full Procrustes mean is the leading eigenfunction of the conditional covariance. To estimate this covariance surface, we propose a novel deep conditional covariance smoother with modality-specific encoders - e.g. splines for scalar covariates and convolutional networks for images, which classical spline smoothers cannot accommodate. Our model is by construction invariant to the translation, rotation and scaling of the input curves and handles sparsely and irregularly sampled curves. We further provide an algorithm for elastic mean estimation that also removes parametrisation by iterating covariance smoothing, rotational alignment and parametrisation alignment. We illustrate the method on simulated outlines with known conditional mean and multimodal covariates, and give a first application to hippocampal outlines from the ADNI cohort, recovering covariate effects consistent with the literature. Code is available at https://github.com/mpff/dnn-shapes.