# A Multi-Level, Multi-Scale Visual Analytics Approach to Assessment of Multifidelity HPC Systems

arXiv: https://arxiv.org/abs/2306.09457 · HTML: https://mlsanity.com/paper/2306.09457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 9 of 9)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [A Survey on Automated Log Analysis for Reliability Engineering](https://mlsanity.com/paper/2009.07237/) (2020)
- [Extracting spatial-temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition](https://mlsanity.com/paper/1409.5496/) (2014)
- [ConceptScope: Organizing and Visualizing Knowledge in Documents based on Domain Ontology](https://mlsanity.com/paper/2003.05108/) (2020)
- [A Visual Analytics Framework for Reviewing Streaming Performance Data](https://mlsanity.com/paper/2001.09399/) (2020)
- [DCDB Wintermute: Enabling Online and Holistic Operational Data Analytics on HPC Systems](https://mlsanity.com/paper/1910.06156/) (2019)
- [A Visual Analytics Approach for Hardware System Monitoring with Streaming Functional Data Analysis](https://mlsanity.com/paper/2011.13079/) (2020)
- [Scalable Comparative Visualization of Ensembles of Call Graphs](https://mlsanity.com/paper/2007.01395/) (2020)

## Abstract (arXiv)

The ability to monitor and interpret of hardware system events and behaviors are crucial to improving the robustness and reliability of these systems, especially in a supercomputing facility. The growing complexity and scale of these systems demand an increase in monitoring data collected at multiple fidelity levels and varying temporal resolutions. In this work, we aim to build a holistic analytical system that helps make sense of such massive data, mainly the hardware logs, job logs, and environment logs collected from disparate subsystems and components of a supercomputer system. This end-to-end log analysis system, coupled with visual analytics support, allows users to glean and promptly extract supercomputer usage and error patterns at varying temporal and spatial resolutions. We use multiresolution dynamic mode decomposition (mrDMD), a technique that depicts high-dimensional data as correlated spatial-temporal variations patterns or modes, to extract variation patterns isolated at specified frequencies. Our improvements to the mrDMD algorithm help promptly reveal useful information in the massive environment log dataset, which is then associated with the processed hardware and job log datasets using our visual analytics system. Furthermore, our system can identify the usage and error patterns filtered at user, project, and subcomponent levels. We exemplify the effectiveness of our approach with two use scenarios with the Cray XC40 supercomputer.