Popular GitHub Repositories for Kv

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

用超强记忆力打造你自己的第二大脑。它就像你的书签专属ChatGPT。使用Chrome扩展程序导入推文或保存网站和内容。

Supercharge Your LLM with the Fastest KV Cache Layer

Unified KV Cache Compression Methods for Auto-Regressive Models

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

LLM KV cache compression made easy

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

Easyreadme helps you simplify README creation and generate visually stunning ones with the help of AI and elegant pre-designed templates.

Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)

[ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection

Full-stack AI chat platform built on Cloudflare using Workers, Durable Objects, KV, and AI Gateway. Features AI chat, Text-to-Speech (TTS), and Speech-to-Text (STT).

Query-agnostic KV cache eviction: 3–4× reduction in memory and 2× decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)

Completion After Prompt Probability. Make your LLM make a choice

Notes about LLaMA 2 model