# Ablation Studies in Artificial Neural Networks

arXiv: https://arxiv.org/abs/1901.08644 · HTML: https://mlsanity.com/paper/1901.08644/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 30 of 60)

- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [How to avoid machine learning pitfalls: a guide for academic researchers](https://mlsanity.com/paper/2108.02497/) (2021)
- [Mind the gap: Challenges of deep learning approaches to Theory of Mind](https://mlsanity.com/paper/2203.16540/) (2022)
- [A Philosophical Introduction to Language Models - Part II: The Way Forward](https://mlsanity.com/paper/2405.03207/) (2024)
- [EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification](https://mlsanity.com/paper/2101.10932/) (2021)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [NERIF: GPT-4V for Automatic Scoring of Drawn Models](https://mlsanity.com/paper/2311.12990/) (2023)
- [Multi-modal Program Inference: a Marriage of Pre-trainedLanguage Models and Component-based Synthesis](https://mlsanity.com/paper/2109.02445/) (2021)
- [Perceptual Musical Features for Interpretable Audio Tagging](https://mlsanity.com/paper/2312.11234/) (2023)
- [Enriching Tabular Data with Contextual LLM Embeddings: A Comprehensive Ablation Study for Ensemble Classifiers](https://mlsanity.com/paper/2411.01645/) (2024)
- [Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks](https://mlsanity.com/paper/2511.19265/) (2025)
- [Two-step adversarial debiasing with partial learning -- medical image case-studies](https://mlsanity.com/paper/2111.08711/) (2021)
- [On the special role of class-selective neurons in early training](https://mlsanity.com/paper/2305.17409/) (2023)
- [Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation](https://mlsanity.com/paper/2406.12516/) (2024)
- [Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control](https://mlsanity.com/paper/2607.14890/) (2026)
- [Evaluation in Neural Style Transfer: A Review](https://mlsanity.com/paper/2401.17109/) (2024)
- [Post-hurricane building damage assessment using street-view imagery and structured data: A multi-modal deep learning approach](https://mlsanity.com/paper/2404.07399/) (2024)
- [Investigating Neuron Ablation in Attention Heads: The Case for Peak Activation Centering](https://mlsanity.com/paper/2408.17322/) (2024)
- [Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications](https://mlsanity.com/paper/2603.22473/) (2026)
- [Enhancing object detection robustness: A synthetic and natural perturbation approach](https://mlsanity.com/paper/2304.10622/) (2023)
- [Leveraging graph neural networks for supporting Automatic Triage of Patients](https://mlsanity.com/paper/2403.07038/) (2024)
- [Machine Learning Analysis of Anomalous Diffusion](https://mlsanity.com/paper/2412.01393/) (2024)
- [Rethinking Inter-LoRA Orthogonality in Adapter Merging: Insights from Orthogonal Monte Carlo Dropout](https://mlsanity.com/paper/2510.03262/) (2025)
- [Advances in Classifying the Stages of Diabetic Retinopathy Using Convolutional Neural Networks in Low Memory Edge Devices](https://mlsanity.com/paper/2106.01739/) (2021)
- [Rogue Cell: Adversarial Attack and Defense in Untrusted O-RAN Setup Exploiting the Traffic Steering xApp](https://mlsanity.com/paper/2505.01816/) (2025)
- [AblationBench: Evaluating Automated Planning of Ablations in Empirical AI Research](https://mlsanity.com/paper/2507.08038/) (2025)
- [Countermind: A Multi-Layered Security Architecture for Large Language Models](https://mlsanity.com/paper/2510.11837/) (2025)
- [PublicAgent: Multi-Agent Design Principles From an LLM-Based Open Data Analysis Framework](https://mlsanity.com/paper/2511.03023/) (2025)
- [Domain-guided Machine Learning for Remotely Sensed In-Season Crop Growth Estimation](https://mlsanity.com/paper/2106.13323/) (2021)
- [Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding](https://mlsanity.com/paper/2301.03765/) (2023)

## Abstract (arXiv)

Ablation studies have been widely used in the field of neuroscience to tackle complex biological systems such as the extensively studied Drosophila central nervous system, the vertebrate brain and more interestingly and most delicately, the human brain. In the past, these kinds of studies were utilized to uncover structure and organization in the brain, i.e. a mapping of features inherent to external stimuli onto different areas of the neocortex. considering the growth in size and complexity of state-of-the-art artificial neural networks (ANNs) and the corresponding growth in complexity of the tasks that are tackled by these networks, the question arises whether ablation studies may be used to investigate these networks for a similar organization of their inner representations. In this paper, we address this question and performed two ablation studies in two fundamentally different ANNs to investigate their inner representations of two well-known benchmark datasets from the computer vision domain. We found that features distinct to the local and global structure of the data are selectively represented in specific parts of the network. Furthermore, some of these representations are redundant, awarding the network a certain robustness to structural damages. We further determined the importance of specific parts of the network for the classification task solely based on the weight structure of single units. Finally, we examined the ability of damaged networks to recover from the consequences of ablations by means of recovery training. We argue that ablations studies are a feasible method to investigate knowledge representations in ANNs and are especially helpful to examine a networks robustness to structural damages, a feature of ANNs that will become increasingly important for future safety-critical applications.