Cache Aside 相关的热门 GitHub AI项目仓库

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

Supercharge Your LLM with the Fastest KV Cache Layer

可扩展的多模态检索增强生成平台

用于自回归模型的统一键值缓存压缩方法

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

一个大型语言模型语义缓存系统，旨在通过缓存查询结果对来减少响应时间，从而提升用户体验。

大型语言模型（LLM）笔记，涵盖模型推理、Transformer模型结构以及LLM框架代码分析。

kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.

一步一步完成Llama 3推理过程，掌握核心概念，理解过程推导，并实现代码。

NeurIPS 2023会议论文：H2O：用于高效生成式大语言模型推理的重击预言机

High performance distributed cache system. Built by Rust.

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

?A curated list of Awesome Diffusion Inference Papers with Codes: Sampling, Cache, Quantization, Parallelism, etc.?

Redis 向量库 (RedisVL) —— Redis 的原生 AI Python 客户端

向量存储是一个向量数据库，它允许在浏览器本地存储中对文本文档进行语义相似性搜索。它使用 OpenAI 嵌入将文档转换为向量，并允许基于余弦相似度搜索相似文档。

一套包含内存向量数据库和AI代理的工具集

[Survey] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

块式变换器：用于快速推理的全局到局部语言建模（NeurIPS 2024）

Query-agnostic KV cache eviction: 3–4× reduction in memory and 2× decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)

Persist and reuse KV Cache to speedup your LLM.