# NeuralSens: Sensitivity Analysis of Neural Networks

arXiv: https://arxiv.org/abs/2002.11423 · HTML: https://mlsanity.com/paper/2002.11423/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 14 of 14)

- [A Review of Machine Learning Methods Applied to Structural Dynamics and Vibroacoustic](https://mlsanity.com/paper/2204.06362/) (2022)
- [Towards Long-Range ENSO Prediction with an Explainable Deep Learning Model](https://mlsanity.com/paper/2503.19502/) (2025)
- [How to Train Unstable Looped Tensor Network](https://mlsanity.com/paper/2203.02617/) (2022)
- [Neuron Sensitivity Guided Test Case Selection for Deep Learning Testing](https://mlsanity.com/paper/2307.11011/) (2023)
- [A mathematical certification for positivity conditions in Neural Networks with applications to partial monotonicity and Trustworthy AI](https://mlsanity.com/paper/2406.08525/) (2024)
- [On Machine Learning-Driven Surrogates for Sound Transmission Loss Simulations](https://mlsanity.com/paper/2204.12290/) (2022)
- [Standalone Neural ODEs with Sensitivity Analysis](https://mlsanity.com/paper/2205.13933/) (2022)
- [Metric Tools for Sensitivity Analysis with Applications to Neural Networks](https://mlsanity.com/paper/2305.02368/) (2023)
- [Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processes](https://mlsanity.com/paper/2308.00858/) (2023)
- [BARMPy: Bayesian Additive Regression Models Python Package](https://mlsanity.com/paper/2404.04738/) (2024)
- [nn2poly: An R Package for Converting Neural Networks into Interpretable Polynomials](https://mlsanity.com/paper/2406.01588/) (2024)
- [Improving the Region of Attraction of a Multi-rotor UAV by Estimating Unknown Disturbances](https://mlsanity.com/paper/2409.00257/) (2024)
- [Exploring Channel Distinguishability in Local Neighborhoods of the Model Space in Quantum Neural Networks](https://mlsanity.com/paper/2410.09470/) (2024)
- [Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks](https://mlsanity.com/paper/2605.03598/) (2026)

## Abstract (arXiv)

Neural networks are important tools for data-intensive analysis and are commonly applied to model non-linear relationships between dependent and independent variables. However, neural networks are usually seen as "black boxes" that offer minimal information about how the input variables are used to predict the response in a fitted model. This article describes the pkgNeuralSens package that can be used to perform sensitivity analysis of neural networks using the partial derivatives method. Functions in the package can be used to obtain the sensitivities of the output with respect to the input variables, evaluate variable importance based on sensitivity measures and characterize relationships between input and output variables. Methods to calculate sensitivities are provided for objects from common neural network packages in proglangR, including pkgneuralnet, pkgnnet, pkgRSNNS, pkgh2o, pkgneural, pkgforecast and pkgcaret. The article presents an overview of the techniques for obtaining information from neural network models, a theoretical foundation of how are calculated the partial derivatives of the output with respect to the inputs of a multi-layer perceptron model, a description of the package structure and functions, and applied examples to compare pkgNeuralSens functions with analogous functions from other available proglangR packages.