mlsanityData to 14 Oct 2026

LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation framework

arXiv 2507.04723 ↗First posted 7 Jul 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Long-context processing has become a fundamental capability for large language models~(LLMs). To assess model's long-context performance, numerous long-context evaluation benchmarks have been proposed. However, variations in evaluation settings across these benchmarks lead to inconsistent results, making it difficult to draw reliable comparisons. Besides, the high computational cost of long-context evaluation poses a significant barrier for the community to conduct comprehensive assessments of long-context models. In this paper, we propose LOOM-Scope, a comprehensive and efficient framework for long-context evaluation. LOOM-Scope standardizes evaluation settings across diverse benchmarks, supports deployment of efficient long-context inference acceleration methods, and introduces a holistic yet lightweight benchmark suite to evaluate models comprehensively. Homepage: https://loomscope.github.io

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

33 of 54 references.

References · 33

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  6. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  10. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  11. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  12. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
21 more
  1. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  2. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 215
  3. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  4. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 144
  5. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  6. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
  7. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  8. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 125
  9. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 123
  10. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 119
  11. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 96
  12. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 81
  13. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inferencecited by 75
  14. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferencescited by 58
  15. 2024Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformerscited by 52
  16. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 50
  17. 2024UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMscited by 24
  18. 2025Accurate KV Cache Quantization with Outlier Tokens Tracingcited by 18
  19. 2024Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?cited by 12
  20. 2025TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimizationcited by 10
  21. 2025Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contextscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Revisiting Long-context Modeling from Context Denoising Perspectivecited by 5
  2. 2025LongRM: Revealing and Unlocking the Context Boundary of Reward Modelingcited by 5
  3. 2026Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformerscited by 5
  4. 2026MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Modelscited by 2
  5. 2026Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Systematic Evaluation of Optimization Techniques for Long-Context Language Models
  2. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Models
  3. 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
  4. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
  5. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  6. 2025LongReasonArena: A Long Reasoning Benchmark for Large Language Models
  7. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  8. 2023Retrieval meets Long Context Large Language Models