mlsanityData to 7 Oct 2026

A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset

arXiv 1806.00358 ↗First posted 1 Jun 2018cs.AI cs.CL cs.IRcited by 54

Not on the map. Why?

Abstract (arXiv)

The recent work of Clark et al. introduces the AI2 Reasoning Challenge (ARC) and the associated ARC dataset that partitions open domain, complex science questions into an Easy Set and a Challenge Set. That paper includes an analysis of 100 questions with respect to the types of knowledge and reasoning required to answer them; however, it does not include clear definitions of these types, nor does it offer information about the quality of the labels. We propose a comprehensive set of definitions of knowledge and reasoning types necessary for answering the questions in the ARC dataset. Using ten annotators and a sophisticated annotation interface, we analyze the distribution of labels across the Challenge Set and statistics related to them. Additionally, we demonstrate that although naive information retrieval methods return sentences that are irrelevant to answering the query, sufficient supporting text is often present in the (ARC) corpus. Evaluating with human-selected relevant sentences improves the performance of a neural machine comprehension model by 42 points.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 8; 2024: 8; 2025: 17; 2026: 14.

54 papers.

Sort:

By area:

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  3. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  4. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  5. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  6. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 120
  7. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  8. 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 79
  9. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  10. 2023QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Modelscited by 39
  11. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  12. 2023Efficient Continual Pre-training for Building Domain Specific Large Language Modelscited by 33
38 more
  1. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 27
  2. 2024Can Large Language Models Understand Context?cited by 26
  3. 2025MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methodscited by 26
  4. 2025Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluationcited by 20
  5. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 19
  6. 2024Pruning Foundation Models for High Accuracy without Retrainingcited by 19
  7. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 18
  8. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMscited by 17
  9. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 17
  10. 2023One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Modelscited by 16
  11. 2023ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskscited by 14
  12. 2025DartQuant: Efficient Rotational Distribution Calibration for LLM Quantizationcited by 14
  13. 2022Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformerscited by 11
  14. 2025SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantizationcited by 9
  15. 2026KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Modelscited by 7
  16. 2025DLP: Dynamic Layerwise Pruning in Large Language Modelscited by 6
  17. 2026GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensingcited by 6
  18. 2025DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptationcited by 4
  19. 2025RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantizationcited by 4
  20. 2022Feeding What You Need by Understanding What You Learnedcited by 3
  21. 2025HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Spacecited by 3
  22. 2025PocketLLM: Ultimate Compression of Large Language Models via Meta Networkscited by 3
  23. 2026LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotationcited by 3
  24. 2025HASHIRU: Hierarchical Agent System for Hybrid Intelligent Resource Utilizationcited by 2
  25. 2025BAQ: Efficient Bit Allocation Quantization for Large Language Modelscited by 2
  26. 2025RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Modelscited by 2
  27. 2026FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketchingcited by 2
  28. 2021Semantic Categorization of Social Knowledge for Commonsense Question Answeringcited by 1
  29. 2023On Degrees of Freedom in Defining and Testing Natural Language Understandingcited by 1
  30. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Modelscited by 1
  31. 2026∞-MoE: Generalizing Mixture of Experts to Infinite Expertscited by 1
  32. 2026SimDiff: Depth Pruning via Similarity and Differencecited by 1
  33. 2026ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMscited by 1
  34. 2024PQ-GCN: Enhancing Text Graph Question Classification with Phrase Featurescited by 0
  35. 2025SwiftPrune: Hessian-Free Weight Pruning for Large Language Modelscited by 0
  36. 2026Exclusive Unlearningcited by 0
  37. 2026TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tilingcited by 0
  38. 2026Theory-optimal Quantization Based on Flatnesscited by 0

50 most cited of 54 All 54 (JSON)