# Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction

arXiv: https://arxiv.org/abs/2504.04202 · HTML: https://mlsanity.com/paper/2504.04202/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 5 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Deep Learning with Topological Signatures](https://mlsanity.com/paper/1707.04041/) (2017)
- [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](https://mlsanity.com/paper/2111.10603/) (2021)
- [ImageFolder: Autoregressive Image Generation with Folded Tokens](https://mlsanity.com/paper/2410.01756/) (2024)
- [A Topology Layer for Machine Learning](https://mlsanity.com/paper/1905.12200/) (2019)
- [Topology-Aware Segmentation Using Discrete Morse Theory](https://mlsanity.com/paper/2103.09992/) (2021)
- [SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer](https://mlsanity.com/paper/2412.10958/) (2024)
- [QARV: Quantization-Aware ResNet VAE for Lossy Image Compression](https://mlsanity.com/paper/2302.08899/) (2023)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [Boosting Latent Diffusion with Perceptual Objectives](https://mlsanity.com/paper/2411.04873/) (2024)
- [Epsilon-VAE: Denoising as Visual Decoding](https://mlsanity.com/paper/2410.04081/) (2024)
- [EAGLE: An Edge-Aware Gradient Localization Enhanced Loss for CT Image Reconstruction](https://mlsanity.com/paper/2403.10695/) (2024)
- [Balanced Rate-Distortion Optimization in Learned Image Compression](https://mlsanity.com/paper/2502.20161/) (2025)
- [Geometry-Aware Merge Tree Comparisons for Time-Varying Data with Interleaving Distances](https://mlsanity.com/paper/2107.14373/) (2021)

## Abstract (arXiv)

Mean-squared error is the default objective for training autoencoders, yet compressed reconstructions often depend not only on pointwise accuracy but also on preserving local spatial order. We study whether structural auxiliary losses can improve, rather than trade off against, MSE in finite-capacity autoencoders. We introduce finite-difference sign error (FDSE), a local-order auxiliary objective that penalizes disagreements between the signs of neighboring finite differences in the target and reconstruction. FDSE is simple, architecture-agnostic, and differentiable through smooth sign surrogates. Across four tensor reconstruction tasks, we find that moderate mixtures of MSE and FDSE can substantially reduce validation MSE relative to pure MSE training. In coefficient sweeps, FDSE mixtures reduce validation MSE by 2.3×--7.0× over pure MSE on these tasks, while comparisons with other auxiliary objectives show FDSE to be among the strongest structural objectives tested. The effect is not universal: pure FDSE performs poorly, and gains are largest for coherent spatial fields where local order carries information about the underlying signal. These results suggest that, in compressed-latent reconstruction, appropriately weighted local-structure supervision can guide optimization toward solutions with better pointwise accuracy, rather than merely improving perceptual or structural metrics at MSE's expense.