# An Empirical Study of Observability Limits in Advanced Software Supply Chain Attacks

arXiv: https://arxiv.org/abs/2603.16694 · HTML: https://mlsanity.com/paper/2603.16694/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2026.

## Built on by (most cited first, 3 of 3)

- [DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction](https://mlsanity.com/paper/2608.03591/) (2026)
- [eDySec: A Deep Learning-based Explainable Dynamic Analysis Framework for Detecting Malicious Packages in PyPI Ecosystem](https://mlsanity.com/paper/2604.26219/) (2026)
- [APTInvestBench: Evaluating Autonomous APT Investigation under Varying Telemetry](https://mlsanity.com/paper/2609.38954/) (2026)

## Abstract (arXiv)

Advanced software supply chain (SSC) attacks are increasingly runtime-only and leave fragmented evidence across hosts, services, and build/dependency layers, making any single telemetry stream insufficient for chain reconstruction. Despite this, no existing dataset provides multi-source runtime monitoring data with end-to-end chain-level ground truth for SSC attacks, leaving the observability limits of such attacks poorly understood. We present SynthChain, a multi-source runtime dataset with chain-level ground truth derived from real-world malicious packages and exploit campaigns, and use it to empirically study observability limits in SSC attacks. SynthChain covers seven representative SSC exploit scenarios across PyPI, npm, and C++ supply chains, spanning Windows, Linux, and containerized environments, with annotations for 14 MITRE ATT&CK tactics, 161 techniques, and 2,919 manually verified Indicators of Compromise (IOC) annotations across 22 log files spanning 11 telemetry types. Our observability analysis shows that no single source is chain-complete: even the best single source recovers fewer than 40% of expected attack steps. Fusing just two complementary sources improves reconstruction by roughly (1.6×), but gains depend on which sources are combined rather than how many. We identify three systematic failure modes---missing-phase gaps, attribution breaks, and ambiguity---and derive telemetry planning guidelines that do not require prior knowledge of specific attacks. A preliminary sensitivity analysis confirms that the multi-source advantage persists under reduced per-source sampling rates. The corpus (approximately 0.59 million events) is released with ground truth and artifacts to support reproducible evaluation of runtime SSC defenses.