# Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding

arXiv: https://arxiv.org/abs/2508.12590 · HTML: https://mlsanity.com/paper/2508.12590/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 13 of 13)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [DistillSpec: Improving Speculative Decoding via Knowledge Distillation](https://mlsanity.com/paper/2310.08461/) (2023)
- [Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities](https://mlsanity.com/paper/2309.16739/) (2023)
- [Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters](https://mlsanity.com/paper/2406.12335/) (2024)
- [Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding](https://mlsanity.com/paper/2505.16652/) (2025)
- [QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache](https://mlsanity.com/paper/2502.10424/) (2025)
- [Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention](https://mlsanity.com/paper/2110.09310/) (2021)
- [Uncertainty-Aware Hybrid Inference with On-Device Small and Remote Large Language Models](https://mlsanity.com/paper/2412.12687/) (2024)
- [LATTE: Low-Precision Approximate Attention with Head-wise Trainable Threshold for Efficient Transformer](https://mlsanity.com/paper/2404.07519/) (2024)

## Built on by (most cited first, 1 of 1)

- [Action Deviation-Aware Inference for Low-Latency Wireless Robots](https://mlsanity.com/paper/2510.02851/) (2025)

## Abstract (arXiv)

To address the growing demand for on-device LLM inference in resource-constrained environments, hybrid language models (HLM) have emerged, combining lightweight local models with powerful cloud-based LLMs. Recent studies on HLM have primarily focused on improving accuracy and latency, while often overlooking communication and energy efficiency. We propose a token-level filtering mechanism for an energy-efficient importance- and uncertainty-aware HLM inference that leverages both epistemic uncertainty and attention-based importance. Our method opportunistically uploads only informative tokens, reducing LLM usage and communication costs. Experiments with TinyLlama-1.1B and LLaMA-2-7B demonstrate that our method achieves up to 87.5% BERT Score and token throughput of 0.37 tokens/sec while saving the energy consumption by 40.7% compared to standard HLM. Furthermore, compared to our previous U-HLM baseline, our method improves BERTScore from 85.8% to 87.0%, energy savings from 31.6% to 43.6%, and throughput from 0.36 to 0.40. This approach enables an energy-efficient and accurate deployment of LLMs in bandwidth-constrained edge environments.