mlsanityData to 7 Oct 2026

Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge

arXiv 2604.22939 ↗First posted 24 Apr 2026cs.CL cs.AI cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialized, non-generative tasks. We attribute this performance bottleneck to the LLMs' knowledge expression mechanism, rather than to deficiencies in knowledge acquisition. To address this, we propose Self-Knowledge Re-expression (SKR), a novel, task-agnostic adaptation method. SKR transforms the LLM's output from generic token generation to highly efficient, task-specific expression. SKR is a fully local method that uses only unannotated data, requiring neither human supervision nor model distillation. Experiments on a large financial document dataset demonstrate substantial improvements: over 40% in Recall@1 for information retrieval tasks, over 76% reduction in object detection latency, and over 33% increase in anomaly detection AUPRC. Our results on the MMDocRAG dataset surpass those of leading retrieval models by at least 12.6%.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

48 of 65 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
36 more
  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2025Gemma 3 Technical Reportcited by 1,686
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  5. 2024Phi-4 Technical Reportcited by 675
  6. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  7. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  8. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  9. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  10. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  11. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  12. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  13. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  14. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  15. 2024The pitfalls of next-token predictioncited by 156
  16. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 123
  17. 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
  18. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  19. 2023From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Modelscited by 96
  20. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
  21. 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
  22. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  23. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  24. 2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 47
  25. 2024Jina CLIP: Your CLIP Model Is Also Your Text Retrievercited by 46
  26. 2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
  27. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 33
  28. 2024MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problemscited by 24
  29. 2024SciMMIR: Benchmarking Scientific Multi-modal Information Retrievalcited by 22
  30. 2024Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Modelscited by 15
  31. 2024VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detectioncited by 13
  32. 2025Closing the Modality Gap for Mixed Modality Searchcited by 11
  33. 2024MANA-Net: Mitigating Aggregated Sentiment Homogenization with News Weighting for Enhanced Market Predictioncited by 6
  34. 2024Modeling News Interactions and Influence for Financial Market Predictioncited by 6
  35. 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrievalcited by 5
  36. 2025Vector Ontologies as an LLM world view extraction methodcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Future Confidence Distillation in Large Language Modelscited by 1