mlsanityData to 7 Oct 2026

CQ-CiM: Hardware-Aware Embedding Shaping for Robust CiM-Based Retrieval

arXiv 2602.20083 ↗First posted 23 Feb 2026cs.ET cs.ARcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Deploying Retrieval-Augmented Generation (RAG) on edge devices is in high demand, but is hindered by the latency of massive data movement and computation on traditional architectures. Compute-in-Memory (CiM) architectures address this bottleneck by performing vector search directly within their crossbar structure. However, CiM's adoption for RAG is limited by a fundamental ``representation gap,'' as high-precision, high-dimension embeddings are incompatible with CiM's low-precision, low-dimension array constraints. This gap is compounded by the diversity of CiM implementations (e.g., SRAM, ReRAM, FeFET), each with unique designs (e.g., 2-bit cells, 512x512 arrays). Consequently, RAG data must be naively reshaped to fit each target implementation. Current data shaping methods handle dimension and precision disjointly, which degrades data fidelity. This not only negates the advantages of CiM for RAG but also confuses hardware designers, making it unclear if a failure is due to the circuit design or the degraded input data. As a result, CiM adoption remains limited. In this paper, we introduce CQ-CiM, a unified, hardware-aware data shaping framework that jointly learns Compression and Quantization to produce CiM-compatible low-bit embeddings for diverse CiM designs. To the best of our knowledge, this is the first work to shape data for comprehensive CiM usage on RAG.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 50 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

20 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures7 May

Other references · 19

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  10. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  11. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  12. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
7 more
  1. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  2. 2013A Theoretical Analysis of NDCG Type Ranking Measurescited by 119
  3. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  4. 2025KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Modelcited by 36
  5. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  6. 2024Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devicescited by 17
  7. 2024A Review of SRAM-based Compute-in-Memory Circuitscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DIRC-RAG: Accelerating Edge RAG with Robust High-Density and High-Loading-Bandwidth Digital In-ReRAM Computation
  2. 2026CiMRAG: CiM-Aware Domain-Adaptive and Noise-Resilient Retrieval-Augmented Generation for Edge-Based LLMs
  3. 2024EdgeRAG: Online-Indexed RAG for Edge Devices
  4. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
  5. 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite
  6. 2026HAVEN: High-Bandwidth Flash Augmented Vector Engine for Large-Scale Approximate Nearest-Neighbor Search Acceleration
  7. 2026RAGe: A Retrieval-Augmented Generation Evaluation Framework
  8. 2025RAGDoll: Efficient Offloading-based Online RAG System on a Single GPU