Unknown-Traffic Detection, Calibration and Shortcut Reliance in Distilled Encrypted-Traffic Classifiers over One Year
On the map: Knowledge distillation → Federated, robust and private distillation also fits: General knowledge distillation methods
Abstract (arXiv)
Knowledge distillation is the standard way to compress encrypted-traffic classifiers for the edge, and almost all such work judges students by accuracy alone. We ask what else a student inherits: unknown-traffic detection, calibration, shortcut reliance, and whether any survives a year of drift. Resemblance proves little on its own, since soft targets also regularise. We therefore distil one 101k-parameter student from two teachers of equal accuracy but different construction, a five-member ensemble and a single wider model, so that following one rather than the other is attributable to it. The design was pre-registered before any test result was seen. We tested ten hypotheses on CESNET-TLS-Year22, a year of real TLS traffic, across 18 test windows over 35 weeks. Two are supported: a student's per-flow unknown-scores shift toward its own teacher, but only at a conventional temperature, not the accuracy-optimal one; and a shortcut-reliant teacher passes its over-confidence to a student that never sees the feature. The drift prediction is reversed under both scores, the gap narrowing rather than widening and the student overtaking under the energy score in two of three replicates, as is the prediction that such a teacher harms its student's detection, which improves slightly. Shortcut reliance is set by model size, not distillation. Under the logit-based scores nothing else transfers: distillation beats neither a temperature-scaled direct student nor label smoothing. Exploratory analysis shows this turns on the scoring rule: with a feature-space detector the teacher detects unknown traffic 0.073 AUROC better than the direct student, where the energy score sees 0.000, and the conventional-temperature student inherits most of it. Label smoothing, with no teacher, recovers more. Distillation transfers the teacher's habits; what looks like an inherited ability is available without one.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 43 of its 49 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
43 of 49 references.
References · 43
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2019Similarity-Preserving Knowledge Distillationcited by 516
- 2021Does Knowledge Distillation Really Work?cited by 190
- 2020Characterising Bias in Compressed Modelscited by 167
- 2019Ensemble Distribution Distillationcited by 158
- 2022ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classificationcited by 66
- 2022What Knowledge Gets Distilled in Knowledge Distillation?cited by 27
- 2021Teacher's pet: understanding and mitigating biases in distillationcited by 21
- 2021Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targetscited by 19
- 2022Fine-grained TLS services classification with reject optioncited by 14
- 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
- 2022Self-Distribution Distillation: Efficient Uncertainty Estimationcited by 11
31 more
- 2024Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix Itcited by 11
- 2022What Do Compressed Multilingual Machine Translation Models Forget?cited by 9
- 2025Who Taught You That? Tracing Teachers in Model Distillationcited by 8
- 2025Credal Ensemble Distillation for Uncertainty Quantificationcited by 8
- 2026Antidistillation Fingerprintingcited by 6
- 2020Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networkscited by 5
- 2023Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertaintiescited by 5
- 2026When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillationcited by 5
- 2025When Simple Model Just Works: Is Network Traffic Classification in Crisis?cited by 4
- 2025The Role of Teacher Calibration in Knowledge Distillationcited by 4
- 2025Knowledge Distillation Detection for Open-weights Modelscited by 4
- 2025Preventing Shortcut Learning in Medical Image Analysis through Intermediate Layer Knowledge Distillation from Specialist Teacherscited by 4
- 2024Adaptive Label Smoothing for Out-of-Distribution Detectioncited by 2
- 2026Trust the uncertain teacher: distilling dark knowledge via calibrated uncertaintycited by 2
- 2026ResAware: Cross-Environment Website Fingerprinting via Resource-Privileged Distillationcited by 2
- 2024MERLOT: A Distilled LLM-based Mixture-of-Experts Framework for Scalable Encrypted Traffic Classificationcited by 1
- 2025Distillation-Enhanced Clustering Acceleration for Encrypted Traffic Classificationcited by 1
- 2025A Functional Perspective on Knowledge Distillation in Neural Networkscited by 1
- 2026Bias in the Shadows: Explore Shortcuts in Encrypted Network Traffic Classificationcited by 1
- 2026Mean Masked Autoencoder with Flow-Mixing for Encrypted Traffic Classificationcited by 1
- 2026Knowledge Distillation Must Account for What It Losescited by 1
- 2026UniAlign: A Model-Agnostic Framework for Robust Network Traffic Classification under Distribution Shiftscited by 1
- 2026Invariant Gradient Alignment for Robust Reasoning Distillationcited by 1
- 2026Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictionscited by 1
- 2026Open-World Darknet Traffic Recognition Under Leave-One-Service-Out Evaluationcited by 1
- 2026TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenancecited by 1
- 2026Anti-Shortcut Distillation via Temporal Negative Knowledge Transfercited by 1
- 2026CipherSight: Robust Website Fingerprinting via Record-Resource Semantic Supervision under Distribution Shiftscited by 1
- 2026Pruned Traffic Trees: Native Semantic Compression with a Protocol-Structured Model Family for Encrypted Traffic Classificationcited by 1
- 2026The Colossus with Feet of Clay: Debunking Encrypted Traffic Classifiers under PQC Evolutioncited by 1
- 2026SoK: Where Do Flow Labels Come From? Auditing Label Provenance in Encrypted Traffic Benchmarkscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026TDDM-Melatt: A Decoupled Memory and Diffusion Framework for Generalizable Encrypted Traffic Classification
- 2026TrafficMoE: Heterogeneity-aware Mixture of Experts for Encrypted Traffic Classification
- 2026Multimodal Reasoning with LLM for Encrypted Traffic Interpretation: A Benchmark
- 2026MalMoE: Mixture-of-Experts Enhanced Encrypted Malicious Traffic Detection Under Graph Drift
- 2026Treat Traffic Like Trees: A Semantic-Preserving Hierarchical Graph-Based Expert Framework for Encrypted Traffic Analysis
- 2026NetMamba+: A Framework of Pre-trained Models for Efficient and Accurate Network Traffic Classification
- 2025MalRAG: A Retrieval-Augmented LLM Framework for Open-set Malicious Traffic Identification
- 2026ActiveFlowMark: Assessing Tor Anonymity under Active Bandwidth Watermarking ↗