# Gappy local conformal auto-encoders for heterogeneous data fusion: in praise of rigidity

arXiv: https://arxiv.org/abs/2312.13155 · HTML: https://mlsanity.com/paper/2312.13155/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 13 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 3 of 3)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [LOCA: LOcal Conformal Autoencoder for standardized data coordinates](https://mlsanity.com/paper/2004.07234/) (2020)

## Built on by (most cited first, 2 of 2)

- [On Learning what to Learn: heterogeneous observations of dynamics and establishing (possibly causal) relations among them](https://mlsanity.com/paper/2406.06812/) (2024)
- [Conformal Disentanglement and Latent-Space Curation: A Neural Framework for Perspective Synthesis, Differentiation and Targeted Generation](https://mlsanity.com/paper/2408.15344/) (2024)

## Abstract (arXiv)

Fusing measurements from multiple, heterogeneous, partial sources, observing a common object or process, poses challenges due to the increasing availability of numbers and types of sensors. In this work we propose, implement and validate an end-to-end computational pipeline in the form of a multiple-auto-encoder neural network architecture for this task. The inputs to the pipeline are several sets of partial observations, and the result is a globally consistent latent space, harmonizing (rigidifying, fusing) all measurements. The key enabler is the availability of multiple slightly perturbed measurements of each instance:, local measurement, "bursts", that allows us to estimate the local distortion induced by each instrument. We demonstrate the approach in a sequence of examples, starting with simple two-dimensional data sets and proceeding to a Wi-Fi localization problem and to the solution of a "dynamical puzzle" arising in spatio-temporal observations of the solutions of Partial Differential Equations.