# Explainable Multi-View Deep Networks Methodology for Experimental Physics

arXiv: https://arxiv.org/abs/2308.08206 · HTML: https://mlsanity.com/paper/2308.08206/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [A Survey on Multi-view Learning](https://mlsanity.com/paper/1304.5634/) (2013)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions](https://mlsanity.com/paper/2107.11508/) (2021)
- [State-of-the-Art Review of Design of Experiments for Physics-Informed Deep Learning](https://mlsanity.com/paper/2202.06416/) (2022)
- [Explainability Techniques for Chemical Language Models](https://mlsanity.com/paper/2305.16192/) (2023)
- [An End-to-End Computer Vision Methodology for Quantitative Metallography](https://mlsanity.com/paper/2104.11159/) (2021)
- [Complete CVDL Methodology for Investigating Hydrodynamic Instabilities](https://mlsanity.com/paper/2004.03374/) (2020)
- [Determining HEDP Foams' Quality with Multi-View Deep Learning Classification](https://mlsanity.com/paper/2208.07196/) (2022)

## Abstract (arXiv)

Physical experiments often involve multiple imaging representations, such as X-ray scans and microscopic images. Deep learning models have been widely used for supervised analysis in these experiments. Combining different image representations is frequently required to analyze and make a decision properly. Consequently, multi-view data has emerged - datasets where each sample is described by views from different angles, sources, or modalities. These problems are addressed with the concept of multi-view learning. Understanding the decision-making process of deep learning models is essential for reliable and credible analysis. Hence, many explainability methods have been devised recently. Nonetheless, there is a lack of proper explainability in multi-view models, which are challenging to explain due to their architectures. In this paper, we suggest different multi-view architectures for the vision domain, each suited to another problem, and we also present a methodology for explaining these models. To demonstrate the effectiveness of our methodology, we focus on the domain of High Energy Density Physics (HEDP) experiments, where multiple imaging representations are used to assess the quality of foam samples. We apply our methodology to classify the foam samples quality using the suggested multi-view architectures. Through experimental results, we showcase the improvement of accurate architecture choice on both accuracy - 78% to 84% and AUC - 83% to 93% and present a trade-off between performance and explainability. Specifically, we demonstrate that our approach enables the explanation of individual one-view models, providing insights into the decision-making process of each view. This understanding enhances the interpretability of the overall multi-view model. The sources of this work are available at: https://github.com/Scientific-Computing-Lab-NRCN/Multi-View-Explainability.