Popular GitHub Repositories for Gptq

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

Advanced Quantization Algorithm for LLMs/VLMs.

Large Language Models for All, ? Cult and More, Stay in touch !

? X—LLM: Cutting Edge & Easy LLM Finetuning

Run any Large Language Model behind a unified API

? Lightweight OpenAI drop-in replacement for Kubernetes

A guide about how to use GPTQ models with langchain

zero零训练llm调参

Chat to LLaMa 2 that also provides responses with reference documents over vector database. Locally available model using GPTQ 4bit quantization.

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

ChatSakura：Open-source multilingual conversational model.（开源多语言对话大模型）

This repository is for profiling, extracting, visualizing and reusing generative AI weights to hopefully build more accurate AI models and audit/scan weights at rest to identify knowledge domains for risk(s).

A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system

Conversation AI model for open domain dialogs

This project will develop a NEPSE chatbot using an open-source LLM, incorporating sentence transformers, vector database and reranking.

NAACL 论文《量化如何影响大型语言模型的置信度？》的代码

LLM quantization techniques: absmax, zero-point, GPTQ and GGUF

Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models. Supports Transformers, AWQ, GPTQ, llama.cpp (GGUF), Llama models

Effortlessly quantize, benchmark, and publish Hugging Face models with cross-platform support for CPU/GPU. Reduce model size by 75% while maintaining performance.