mlsanityData to 14 Oct 2026

FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models

arXiv 2508.10020 ↗First posted 7 Aug 2025cs.CL cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Enhancing LLM reasoning in federated settings is nontrivial due to stringent computational, communication, and privacy constraints, especially in healthcare, where clinically consequential decisions require not only accuracy but also interpretable, auditable rationales to meet safety, accountability, and regulatory requirements. Conventional federated fine-tuning largely imitates final answers rather than cultivating step-by-step reasoning, often relying on privacy-sensitive centralized distillation and still incurring substantial communication overhead. We address this gap with textbfours, a federated reasoning framework that combines lightweight chain-of-thought resampling with a compact discriminator for selection, and client-aware LoRA stacking with weighted classifier aggregation to accommodate heterogeneity while reducing aggregation noise and communication; clients generate candidate chains and supervision locally, and only lightweight modules are aggregated on the server. Experiments on medical reasoning benchmarks show consistent gains under tight resource budgets while keeping data local and respecting privacy, offering an interpretable and resource-efficient solution. Our code is made publicly available at https://github.com/DIaacKr/FedCoT

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 33 of its 64 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

33 of 64 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation27 Feb
  2. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling30 Aug
  3. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug

Other references · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  10. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  11. 2019Fair Resource Allocation in Federated Learningcited by 394
  12. 2022Teaching Small Language Models to Reasoncited by 284
18 more
  1. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  2. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  3. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
  4. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 124
  5. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
  6. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 105
  7. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  8. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 44
  9. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 36
  10. 2025A Survey on Federated Fine-tuning of Large Language Modelscited by 24
  11. 2024Learning to Maximize Mutual Information for Chain-of-Thought Distillationcited by 23
  12. 2024FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Modelscited by 19
  13. 2023Enhancing Small Medical Learners with Privacy-preserving Contextual Promptingcited by 11
  14. 2020A Theoretical Perspective on Differentially Private Federated Multi-task Learningcited by 8
  15. 2023Trustworthy Federated Learning: A Surveycited by 4
  16. 2024MIDGARD: Self-Consistency Using Minimum Description Length for Structured Commonsense Reasoningcited by 3
  17. 2025DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Modelscited by 3
  18. 2024A Comprehensive Survey on Joint Resource Allocation Strategies in Federated Edge Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Federation over Text: Insight Sharing for Multi-Agent Reasoningcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
  2. 2026FERA: Uncertainty-Aware Federated Reasoning for Large Language Models
  3. 2026Federated Reasoning Distillation Framework with Model Learnability-Aware Data Allocation
  4. 2025Communication-Aware Knowledge Distillation for Federated LLM Fine-Tuning over Wireless Networks
  5. 2026FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks
  6. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  7. 2025FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
  8. 2026Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions