mlsanityData to 14 Oct 2026

Can abstract concepts from LLM improve SLM performance?

arXiv 2512.19069 ↗First posted 22 Dec 2025cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) excel at diverse tasks, but their deployment on resource-constrained devices remains challenging. Existing methods like quantization, pruning, and distillation can reduce memory footprint but often demand extensive experimentation and careful infrastructure design. Leveraging existing techniques for extracting high-level concepts (represented as steering vectors) from larger models, we investigate their transferability to smaller language models (SLM) during inference. We demonstrate through extensive experimentation that these concepts can be effectively transferred to smaller models, irrespective of their family (e.g., Phi, Llama, Qwen), leading to performance improvements across a wide range of tasks. Furthermore, we introduce inference-time scaling to enhance performance by dynamically adjusting the steering intensity which has resulted in a 7-15% of accuracy improvement for Qwen3-0.6B.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 41 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

36 of 41 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
24 more
  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  2. 2025Gemma 3 Technical Reportcited by 1,824
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2023Steering Language Models With Activation Engineeringcited by 922
  7. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  8. 2021GPT Understands, Toocited by 716
  9. 2024Phi-4 Technical Reportcited by 711
  10. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  11. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  12. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  13. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  14. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  15. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82
  16. 2022GPT Takes the Bar Examcited by 65
  17. 2020Pruning Filter in Filtercited by 47
  18. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 47
  19. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  20. 2025Wanda++: Pruning Large Language Models via Regional Gradientscited by 26
  21. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 22
  22. 2020An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devicescited by 18
  23. 2022Trainability Preserving Neural Pruningcited by 16
  24. 2024L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Modelscited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  2. 2024A Survey on Efficient Inference for Large Language Models
  3. 2023Efficient LLM Inference on CPUs
  4. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
  5. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  6. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
  7. 2025Leveraging the true depth of LLMs
  8. 2023Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models