# Sensitivity Analysis On Loss Landscape

arXiv: https://arxiv.org/abs/2403.01128 · HTML: https://mlsanity.com/paper/2403.01128/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 7 references that are in the map.
- First posted on arXiv 2 Mar 2024.

## Builds on (most cited first, 2 of 2)

- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Feature Gradients: Scalable Feature Selection via Discrete Relaxation](https://mlsanity.com/paper/1908.10382/) (2019)

## Abstract (arXiv)

Gradients can be employed for sensitivity analysis. Here, we leverage the advantages of the Loss Landscape to comprehend which independent variables impact the dependent variable. We seek to grasp the loss landscape by utilizing first, second, and third derivatives through automatic differentiation. we know that Spearman's rank correlation coefficient can detect the monotonic relationship between two variables. However, I have found that second-order gradients, with certain configurations and parameters, provide information that can be visualized similarly to Spearman results, In this approach, we incorporate a loss function with an activation function, resulting in a non-linear pattern. Each exploration of the loss landscape through retraining yields new valuable information. Furthermore, the first and third derivatives are also beneficial, as they indicate the extent to which independent variables influence the dependent variable.