mlsanityData to 7 Oct 2026

NinjaLLM: Fast, Scalable and Cost-effective RAG using Amazon SageMaker and AWS Trainium and Inferentia2

arXiv 2407.12057 ↗First posted 11 Jul 2024cs.CL cs.AIcited by 2

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-augmented generation (RAG) techniques are widely used today to retrieve and present information in a conversational format. This paper presents a set of enhancements to traditional RAG techniques, focusing on large language models (LLMs) fine-tuned and hosted on AWS Trainium and Inferentia2 AI chips via SageMaker. These chips are characterized by their elasticity, affordability, and efficient performance for AI compute tasks. Besides enabling deployment on these chips, this work aims to improve tool usage, add citation capabilities, and mitigate the risks of hallucinations and unsafe responses due to context bias. We benchmark our RAG system's performance on the Natural Questions and HotPotQA datasets, achieving an accuracy of 62% and 59% respectively, exceeding other models such as DBRX and Mixtral Instruct.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 9 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

6 of 9 references.

References · 6

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023LIMA: Less Is More for Alignmentcited by 1,080
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  2. 2025NeuronMLP: Efficient LLM Inference via Singular Value Decomposition Compression and Tiling on AWS Trainiumcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  2. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  3. 2024ERATTA: Extreme RAG for Table To Answers with Large Language Models
  4. 2025Comparing the Performance of LLMs in RAG-based Question-Answering: A Case Study in Computer Science Literature
  5. 2024The Power of Noise: Redefining Retrieval for RAG Systems
  6. 2024SFR-RAG: Towards Contextually Faithful LLMs
  7. 2025A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparent
  8. 2024Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup