# MoE-Enhanced Explainable Deep Manifold Transformation for Complex Data Embedding and Visualization

arXiv: https://arxiv.org/abs/2410.19504 · HTML: https://mlsanity.com/paper/2410.19504/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 36 of its 81 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 36)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Online Deep Clustering for Unsupervised Representation Learning](https://mlsanity.com/paper/2006.10645/) (2020)
- [Visualizing Large-scale and High-dimensional Data](https://mlsanity.com/paper/1602.00370/) (2016)
- [Interpretability of machine learning based prediction models in healthcare](https://mlsanity.com/paper/2002.08596/) (2020)
- [AutoMix: Unveiling the Power of Mixup for Stronger Classifiers](https://mlsanity.com/paper/2103.13027/) (2021)
- [The Numerical Stability of Hyperbolic Representation Learning](https://mlsanity.com/paper/2211.00181/) (2022)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [DASVDD: Deep Autoencoding Support Vector Data Descriptor for Anomaly Detection](https://mlsanity.com/paper/2106.05410/) (2021)
- [Hyperbolic Representation Learning: Revisiting and Advancing](https://mlsanity.com/paper/2306.09118/) (2023)
- [DimReader: Axis lines that explain non-linear projections](https://mlsanity.com/paper/1710.00992/) (2017)
- [DLME: Deep Local-flatness Manifold Embedding](https://mlsanity.com/paper/2207.03160/) (2022)
- [Hierarchical Nearest Neighbor Graph Embedding for Efficient Dimensionality Reduction](https://mlsanity.com/paper/2203.12997/) (2022)
- [GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models](https://mlsanity.com/paper/2406.01627/) (2024)
- [EVNet: An Explainable Deep Network for Dimension Reduction](https://mlsanity.com/paper/2211.15478/) (2022)
- [DINE: Dimensional Interpretability of Node Embeddings](https://mlsanity.com/paper/2310.01162/) (2023)
- [DiffAug: Enhance Unsupervised Contrastive Learning with Domain-Knowledge-Free Diffusion-based Data Augmentation](https://mlsanity.com/paper/2309.07909/) (2023)

## Built on by (most cited first, 3 of 3)

- [Breaking Thought Patterns: A Multi-Dimensional Reasoning Framework for LLMs](https://mlsanity.com/paper/2506.13192/) (2025)
- [Anatomy-R1: Enhancing Anatomy Reasoning in Multimodal Large Language Models via Anatomical Similarity Curriculum and Group Diversity Augmentation](https://mlsanity.com/paper/2512.19512/) (2025)
- [HDTree: Generative Modeling of Cellular Hierarchies for Robust Lineage Inference](https://mlsanity.com/paper/2506.23287/) (2025)

## Abstract (arXiv)

Dimensionality reduction (DR) plays a crucial role in various fields, including data engineering and visualization, by simplifying complex datasets while retaining essential information. However, achieving both high DR accuracy and strong explainability remains a fundamental challenge, especially for users dealing with high-dimensional data. Traditional DR methods often face a trade-off between precision and transparency, where optimizing for performance can lead to reduced explainability, and vice versa. This limitation is especially prominent in real-world applications such as image, tabular, and text data analysis, where both accuracy and explainability are critical. To address these challenges, this work introduces the MoE-based Explainable Deep Manifold Transformation (DMT-ME). The proposed approach combines a geometry-aware hyperbolic mapper with Mixture of Experts (MoE) models, where sparse expert specialization provides the main representational gain and the hyperbolic component offers an additional refinement for structurally complex data. DMT-ME enhances DR accuracy primarily through MoE-based sparse routing and structure-aware matching, while also improving explainability by explicitly linking input data, embedding outcomes, and key features through the MoE structure. Extensive experiments demonstrate that DMT-ME consistently achieves superior performance in both DR accuracy and model explainability, making it a robust solution for complex data analysis. The code is available at https://github.com/zangzelin/code_dmtme.