# HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds

arXiv: https://arxiv.org/abs/2605.10536 · HTML: https://mlsanity.com/paper/2605.10536/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 11 of 11)

- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Improving Dictionary Learning with Gated Sparse Autoencoders](https://mlsanity.com/paper/2404.16014/) (2024)
- [SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability](https://mlsanity.com/paper/2503.09532/) (2025)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures](https://mlsanity.com/paper/2506.01197/) (2025)
- [Balanced Mixed-Type Tabular Data Synthesis with Diffusion Models](https://mlsanity.com/paper/2404.08254/) (2024)
- [Representation learning to advance multi-institutional studies with electronic health record data from US and France](https://mlsanity.com/paper/2502.08547/) (2025)
- [Generating Clinically Realistic EHR Data via a Hierarchy- and Semantics-Guided Transformer](https://mlsanity.com/paper/2502.20719/) (2025)
- [Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy](https://mlsanity.com/paper/2505.24473/) (2025)

## Abstract (arXiv)

Rare semantic innovations in high-dimensional, mission-critical domains are often obscured by dense background contexts, a challenge we define as textitfeature density conflict. We introduce the Hybrid Hierarchical SAE (HH-SAE) to resolve this by factorizing manifolds into a nested hierarchy of Contextual (L0), Atomic (f1), and Compository (f2) tiers. Evaluating across disparate manifolds, HH-SAE demonstrates superior resolution by ``fracturing'' administrative clinical labels into physiological modes and achieving a peak cross-domain zero-shot AUC of 0.9156 in fraud detection. Path ablation confirms the architecture's structural necessity, revealing a 13.46% utility collapse when contextual subtraction is removed. Finally, knowledge-steered synthesis achieves a +9.9% AUPRC lift over state-of-the-art generators, proving that HH-SAE effectively prioritizes high-order mechanistic innovation over environmental proxies to enable high-precision discovery in high-stakes environments.