mlsanityData to 7 Oct 2026

V-Seek: Accelerating LLM Reasoning on Open-hardware Server-class RISC-V Platforms

arXiv 2503.17422 ↗First posted 21 Mar 2025cs.LG cs.PFcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

The recent exponential growth of Large Language Models (LLMs) has relied on GPU-based systems. However, CPUs are emerging as a flexible and lower-cost alternative, especially when targeting inference and reasoning workloads. RISC-V is rapidly gaining traction in this area, given its open and vendor-neutral ISA. However, the RISC-V hardware for LLM workloads and the corresponding software ecosystem are not fully mature and streamlined, given the requirement of domain-specific tuning. This paper aims at filling this gap, focusing on optimizing LLM inference on the Sophon SG2042, the first commercially available many-core RISC-V CPU with vector processing capabilities. On two recent state-of-the-art LLMs optimized for reasoning, DeepSeek R1 Distill Llama 8B and DeepSeek R1 Distill QWEN 14B, we achieve 4.32/2.29 token/s for token generation and 6.54/3.68 token/s for prompt processing, with a speed up of up 2.9x/3.0x compared to our baseline.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 3 of its 6 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

3 of 6 references.

References · 3

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  2. 2023Efficient LLM Inference on CPUscited by 24
  3. 2024Performance characterisation of the 64-core SG2042 RISC-V CPU for HPCcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
  2. 2025TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems
  3. 2023Efficient LLM inference solution on Intel GPU
  4. 2026RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems
  5. 2025SAIL: SRAM-Accelerated LLM Inference System with Lookup-Table-based GEMV
  6. 2024Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
  7. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  8. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs