KVQuant

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

A curated list for Efficient Large Language Models

PaddleSlim is an open-source library for deep model compression and architecture search.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

A Github Action that automatically compresses JPEGs, PNGs and WebPs in Pull Requests.

[ICLR 2019] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware