Popular GitHub Repositories for Kvcache Compression

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

? Easy-to-use and powerful NLP and LLM library with ? Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including ?Text Classification, ? Neural Search, ? Question Answering, ?? Information Extraction, ? Document Intelligence, ? Sentiment Analysis etc.

SwinIR: Image Restoration Using Swin Transformer (official repository)

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.

Awesome Knowledge Distillation

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.

QualityScaler - image/video AI upscaler app

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

A curated list of neural network pruning resources.

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

? Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms?