# Privacy-Preserving Heterogeneous Multi-LLM Federated Inference for Cognitive Diagnosis

arXiv: https://arxiv.org/abs/2609.02947 · HTML: https://mlsanity.com/paper/2609.02947/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [EdNet: A Large-Scale Hierarchical Dataset in Education](https://mlsanity.com/paper/1912.03072/) (2019)
- [Neural Cognitive Diagnosis for Intelligent Education Systems](https://mlsanity.com/paper/1908.08733/) (2019)
- [Differential Privacy: An Economic Method for Choosing Epsilon](https://mlsanity.com/paper/1402.3329/) (2014)
- [Do LLMs Give Psychometrically Plausible Responses in Educational Assessments?](https://mlsanity.com/paper/2506.09796/) (2025)
- [Knowledge is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis](https://mlsanity.com/paper/2502.05556/) (2025)
- [Differentially Private Federated Learning with Laplacian Smoothing](https://mlsanity.com/paper/2005.00218/) (2020)
- [Trustworthy Privacy-preserving Hierarchical Ensemble and Federated Learning in Healthcare 4.0 with Blockchain](https://mlsanity.com/paper/2305.09209/) (2023)
- [Knowledge Tracing in Programming Education Integrating Students' Questions](https://mlsanity.com/paper/2502.10408/) (2025)

## Abstract (arXiv)

Significant challenges remain in AI-driven educational systems in balancing privacy preservation with accurate cognitive diagnosis. To overcome this, we propose a federated inference framework in which several commercial LLM APIs collaborate without requiring access to raw student data or proprietary model internals. Using multiple federated entities, such as LLaMA-3.3-70B, GPT-4o-mini, and Claude-3-Haiku, our framework builds upon a heterogeneous multi-LLM architecture. The predictions generated by these entities are combined with epsilon-local differential privacy by adding Laplace noise locally to each entity's prediction output before aggregation, while residual-based aggregation mitigates model heterogeneity. Our approach is predicated on an honest-but-curious trust paradigm in which API providers are presumed not to abuse submitted queries, and our differential privacy mechanism shields the published diagnostic results from external inference. We conduct rigorous privacy-utility analysis showing strong privacy guarantees with minimal accuracy loss, and extensive real-world evaluations across three educational benchmarks confirm the framework's practical usability and cross-domain generalizability.