Popular GitHub Repositories for Quantized

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Vector Quantized VAEs - PyTorch Implementation

A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.

An Open-Source Library for Training Binarized Neural Networks

⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.

QKeras: a quantization deep learning library for Tensorflow Keras

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

Offline semantic Text-to-Image and Image-to-Image search on Android powered by quantized state-of-the-art vision-language pretrained CLIP model and ONNX Runtime inference engine

Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

[IJCAI 2022] FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer

A collection of resources and papers on Vector Quantized Variational Autoencoder (VQ-VAE) and its application

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

[CVPR'20] ZeroQ: A Novel Zero Shot Quantization Framework

Generate saved_model, tfjs, tf-trt, EdgeTPU, CoreML, quantized tflite, ONNX, OpenVINO, Myriad Inference Engine blob and .pb from .tflite. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support. Supports inverse quantization of INT8 quantization model.

An acceleration library that supports arbitrary bit-width combinatorial quantization operations

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.

Pytorch implementation of stochastically quantized variational autoencoder (SQ-VAE)

[NeurIPS 2024 Oral?] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.