Popular GitHub Repositories for Product Quantization

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Opiniated RAG for integrating GenAI in your apps ? Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

The best apps in the AI era aren’t solo experiences—they’re collaborative. Liveblocks provides customizable pre?built features to make your product multiplayer, engaging, and AI?ready. All without derailing your roadmap.

Fast inference engine for Transformer models

Vector (and Scalar) Quantization, in Pytorch

Sparsity-aware deep learning inference runtime for CPUs

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

? Accelerate inference and training of ? Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)

Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Open-source Visual programming for backend logic that integrates with existing codebases. Flyde bridges the gap between technical and non-technical team members. Product managers, designers, and backend developers can collaborate on the same visual flows.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

PyTorch native quantization and sparsity for training and inference

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.