mlsanityData to 7 Oct 2026

Efficient LLM inference and serving

12 directions · 3,186 papers · 35 this week

Busiest this week

Most new since Mon 5 Oct

Speculative decoding

522 papers9 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

  1. Speedbumps: Rejection Attacks on Speculative Decoding
  2. Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
  3. Secure Speculative Decoding for Large Language Models
  4. Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution
  5. APEX: Speculate smarter, not deeper
Open →

All directions

Largest first · week from Mon 5 Oct

  1. KV cache optimizationcache, attention, memory, compressionMedium confidence in this grouping: mostly one topic, with blurry edges · About7087Medium confidence in this grouping: mostly one topic, with blurry edges
  2. Speculative decodingdecoding, speculative, speculative decoding, draftHigh confidence in this grouping: the papers form one clear topic · About5229High confidence in this grouping: the papers form one clear topic
  3. Visual token pruning in multimodal modelsvisual, token, tokens, pruningHigh confidence in this grouping: the papers form one clear topic · About5204High confidence in this grouping: the papers form one clear topic
  4. Serving systems and request schedulingllm, serving, llm serving, inferenceHigh confidence in this grouping: the papers form one clear topic · About4020High confidence in this grouping: the papers form one clear topic
  5. Early exit and edge inferenceinference, early, exit, early-exitLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About2150Low confidence in this grouping: the papers may belong to more than one topic, or the group may change
  6. Efficient reasoning and agent inferencereasoning, cache, compression, agentsHigh confidence in this grouping: the papers form one clear topic · About1725High confidence in this grouping: the papers form one clear topic
  7. Diffusion LLM and MoE inferencedecoding, diffusion, parallel, dllmsMedium confidence in this grouping: mostly one topic, with blurry edges · About1562Medium confidence in this grouping: mostly one topic, with blurry edges
  8. Prompt and context compressioncompression, context, prompt, prompt compressionHigh confidence in this grouping: the papers form one clear topic · About1320High confidence in this grouping: the papers form one clear topic
  9. KV cache quantizationquantization, cache, bit, memoryHigh confidence in this grouping: the papers form one clear topic · About1224High confidence in this grouping: the papers form one clear topic
  10. Token pruning and merging for ViTstoken, vision, pruning, transformersHigh confidence in this grouping: the papers form one clear topic · About1082High confidence in this grouping: the papers form one clear topic
  11. Fast autoregressive visual generationgeneration, video, autoregressive, decodingMedium confidence in this grouping: mostly one topic, with blurry edges · About970Medium confidence in this grouping: mostly one topic, with blurry edges
  12. Audio and omni-modal token compressionaudio, token, speech, compressionHigh confidence in this grouping: the papers form one clear topic · About320High confidence in this grouping: the papers form one clear topic
2 new this week aren’t in a direction yet
  1. CHASE: Channel-Aligned Structure Exploitation for Geometry-Aware Model Engineering
  2. Visual Memory Attacks Can Persist Through The KV Cache

Weekly activity

Papers per week, 22 Jun – 5 Oct (last week partial)

  1. DirectionConfidencepapers16 wks
  2. KV cache optimizationMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 12; 29 Jun: 9; 6 Jul: 7; 13 Jul: 7; 20 Jul: 7; 27 Jul: 12; 3 Aug: 17; 10 Aug: 9; 17 Aug: 8; 24 Aug: 12; 31 Aug: 13; 7 Sep: 12; 14 Sep: 13; 21 Sep: 24; 28 Sep: 22; 5 Oct: 7708191
  3. Speculative decodingHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 6; 29 Jun: 6; 6 Jul: 6; 13 Jul: 5; 20 Jul: 6; 27 Jul: 5; 3 Aug: 8; 10 Aug: 6; 17 Aug: 2; 24 Aug: 6; 31 Aug: 5; 7 Sep: 4; 14 Sep: 11; 21 Sep: 8; 28 Sep: 16; 5 Oct: 9522109
  4. Visual token pruning in multimodal modelsHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 8; 29 Jun: 7; 6 Jul: 5; 13 Jul: 4; 20 Jul: 7; 27 Jul: 11; 3 Aug: 15; 10 Aug: 7; 17 Aug: 4; 24 Aug: 10; 31 Aug: 14; 7 Sep: 4; 14 Sep: 5; 21 Sep: 6; 28 Sep: 16; 5 Oct: 4520127
  5. Serving systems and request schedulingHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 5; 29 Jun: 8; 6 Jul: 5; 13 Jul: 3; 20 Jul: 3; 27 Jul: 5; 3 Aug: 7; 10 Aug: 7; 17 Aug: 8; 24 Aug: 1; 31 Aug: 4; 7 Sep: 12; 14 Sep: 9; 21 Sep: 10; 28 Sep: 8; 5 Oct: 040295
  6. Efficient reasoning and agent inferenceHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 2; 29 Jun: 5; 6 Jul: 4; 13 Jul: 4; 20 Jul: 1; 27 Jul: 3; 3 Aug: 5; 10 Aug: 3; 17 Aug: 3; 24 Aug: 4; 31 Aug: 6; 7 Sep: 1; 14 Sep: 5; 21 Sep: 5; 28 Sep: 10; 5 Oct: 517266
  7. Diffusion LLM and MoE inferenceMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 3; 29 Jun: 3; 6 Jul: 2; 13 Jul: 3; 20 Jul: 2; 27 Jul: 4; 3 Aug: 3; 10 Aug: 6; 17 Aug: 2; 24 Aug: 4; 31 Aug: 1; 7 Sep: 3; 14 Sep: 4; 21 Sep: 2; 28 Sep: 9; 5 Oct: 215653
  8. KV cache quantizationHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 3; 29 Jun: 2; 6 Jul: 2; 13 Jul: 1; 20 Jul: 1; 27 Jul: 2; 3 Aug: 2; 10 Aug: 0; 17 Aug: 0; 24 Aug: 2; 31 Aug: 3; 7 Sep: 1; 14 Sep: 2; 21 Sep: 1; 28 Sep: 4; 5 Oct: 412230
  9. Fast autoregressive visual generationMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 1; 29 Jun: 2; 6 Jul: 0; 13 Jul: 1; 20 Jul: 0; 27 Jul: 0; 3 Aug: 0; 10 Aug: 0; 17 Aug: 1; 24 Aug: 2; 31 Aug: 1; 7 Sep: 0; 14 Sep: 2; 21 Sep: 3; 28 Sep: 5; 5 Oct: 09718
  10. Early exit and edge inferenceLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · AboutLow confidence in this grouping: the papers may belong to more than one topic, or the group may changePer week: 22 Jun: 2; 29 Jun: 1; 6 Jul: 0; 13 Jul: 1; 20 Jul: 0; 27 Jul: 2; 3 Aug: 2; 10 Aug: 2; 17 Aug: 0; 24 Aug: 1; 31 Aug: 3; 7 Sep: 3; 14 Sep: 2; 21 Sep: 2; 28 Sep: 3; 5 Oct: 021524
  11. Prompt and context compressionHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 0; 29 Jun: 0; 6 Jul: 3; 13 Jul: 0; 20 Jul: 1; 27 Jul: 2; 3 Aug: 2; 10 Aug: 1; 17 Aug: 1; 24 Aug: 1; 31 Aug: 4; 7 Sep: 1; 14 Sep: 0; 21 Sep: 4; 28 Sep: 1; 5 Oct: 013221
  12. Token pruning and merging for ViTsHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 0; 29 Jun: 2; 6 Jul: 0; 13 Jul: 0; 20 Jul: 0; 27 Jul: 0; 3 Aug: 2; 10 Aug: 2; 17 Aug: 2; 24 Aug: 0; 31 Aug: 1; 7 Sep: 0; 14 Sep: 1; 21 Sep: 1; 28 Sep: 1; 5 Oct: 210814
  13. Audio and omni-modal token compressionHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 1; 29 Jun: 1; 6 Jul: 0; 13 Jul: 0; 20 Jul: 3; 27 Jul: 1; 3 Aug: 5; 10 Aug: 0; 17 Aug: 0; 24 Aug: 1; 31 Aug: 1; 7 Sep: 0; 14 Sep: 0; 21 Sep: 0; 28 Sep: 0; 5 Oct: 03213