Popular GitHub Repositories for Cache Poisoning

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Supercharge Your LLM with the Fastest KV Cache Layer

Scalable Multi-modal RAG platform

Fluid, elastic data abstraction and acceleration for BigData/AI applications in cloud. (Project under CNCF)

Unified KV Cache Compression Methods for Auto-Regressive Models

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

A LLM semantic caching system aiming to enhance user experience by reducing response time via cached query-result pairs.

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

Use ArXiv ChatGuru to talk to research papers. This app uses LangChain, OpenAI, Streamlit, and Redis as a vector database/semantic cache.

LLM KV cache compression made easy

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

This repository contains the code to generate predictions of critical violations at food establishments in Chicago. It also contains the results of an evaluation of the effectiveness of those predictions.

Security and Privacy Risk Simulator for Machine Learning (arXiv:2312.17667)

A curated list of trustworthy deep learning papers. Daily updating...

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.