# Geometry-Preserving Neural Architectures on Manifolds with Boundary

arXiv: https://arxiv.org/abs/2602.03082 · HTML: https://mlsanity.com/paper/2602.03082/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 27 of 27)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Hyperbolic Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.04562/) (2021)
- [Neural Manifold Ordinary Differential Equations](https://mlsanity.com/paper/2006.10254/) (2020)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Universal Approximation Theorems for Differentiable Geometric Deep Learning](https://mlsanity.com/paper/2101.05390/) (2021)
- [End-to-End Feasible Optimization Proxies for Large-Scale Economic Dispatch](https://mlsanity.com/paper/2304.11726/) (2023)
- [Universal Approximation Under Constraints is Possible with Transformers](https://mlsanity.com/paper/2110.03303/) (2021)
- [Neural Ordinary Differential Equations on Manifolds](https://mlsanity.com/paper/2006.06663/) (2020)
- [HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees](https://mlsanity.com/paper/2410.10807/) (2024)
- [Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach](https://mlsanity.com/paper/2106.04941/) (2021)
- [Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2110.13475/) (2021)
- [Riemannian Residual Neural Networks](https://mlsanity.com/paper/2310.10013/) (2023)
- [Topological Obstructions to Autoencoding](https://mlsanity.com/paper/2102.08380/) (2021)
- [Interpreting and Improving Diffusion Models from an Optimization Perspective](https://mlsanity.com/paper/2306.04848/) (2023)
- [Learning Weighted Submanifolds with Variational Autoencoders and Riemannian Variational Autoencoders](https://mlsanity.com/paper/1911.08147/) (2019)
- [Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices](https://mlsanity.com/paper/2306.14064/) (2023)
- [Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs](https://mlsanity.com/paper/2305.08849/) (2023)
- [Why should autoencoders work?](https://mlsanity.com/paper/2310.02250/) (2023)
- [Projected Dynamical Systems on Irregular, Non-Euclidean Domains for Nonlinear Optimization](https://mlsanity.com/paper/1809.04831/) (2018)

## Abstract (arXiv)

A growing number of neural architectures have been proposed to enforce geometric constraints, including projection-based networks, exponential-map updates, constrained output layers, and manifold neural ODEs. We provide a unified framework for these geometry-preserving architectures by organizing them according to where and how constraints are enforced, either throughout the intermediate layers or only at the final output. This perspective reveals several gaps in the existing theory. To address these gaps, we prove high-level approximation theorems for projected neural ODEs, intermediate augmented architectures, and final augmented architectures on prox-regular constraint sets, including smooth manifolds with boundary. Numerical experiments on synthetic dynamics over S^2, the disk, SO(3), together with real-world protein backbone data on SE(3), demonstrate exact feasibility for analytic updates and show that the final augmentation have simpler architecture and outperform in most tasks considered. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Moreover, we also the demonstrate the usefulness of the architectures that enforce non-convex constraints for path planning on manifolds with boundary.