This project performs llama.cpp mixed-precision quantization on DeepSeek-V3-0324, solving the problem of model deployment on edge devices and improving the model's operation efficiency and resource utilization.

Natural Language ProcessingTransformers

This is a random weight generation model based on the DeepSeek-V3 architecture, specifically designed for testing and development purposes. The model uses randomly initialized weights, and the parameter scale is streamlined, making it suitable for rapid prototype development and functional verification.

Natural Language ProcessingTransformers

DeepSeek-V3.2-Exp-AWQ is a quantized version based on the DeepSeek-V3.2-Exp model, which achieves efficient text generation through the vLLM framework. This model introduces the DeepSeek sparse attention mechanism, significantly improving the efficiency of long context training and inference while maintaining the quality of the model output.

Natural Language ProcessingTransformers

The Transformer Library is a powerful natural language processing tool library that provides a rich set of pre-trained models and convenient usage interfaces. It is widely used in NLP tasks such as text generation, classification, and translation, greatly improving development efficiency.

Natural Language ProcessingTransformers

DeepSeek-V3.1 is a large language model that is post-trained based on DeepSeek-V3.1-Base. It has a 128K context length and supports mixed thinking modes, intelligent tool invocation, and code proxy functions. This model performs excellently in multiple benchmark tests, especially showing significant improvements in mathematical reasoning, code generation, and search agent tasks.

Natural Language ProcessingTransformersMultiple Languages

A quantized version in GGUF format based on the DeepSeek-V3.1 model, using the ik_llama.cpp branch for optimal quantization. This model achieves the best perplexity performance under a specific memory footprint and offers various quantization schemes from IQ5_K to IQ1_S to meet different memory and performance requirements.

Natural Language ProcessingGgufOther

This is a quantized version of the DeepSeek-V3.1 model, which uses the imatrix technology of llama.cpp for quantization processing. The aim is to improve the operating efficiency and performance of the model on different hardware. This version offers multiple quantization levels, ranging from high-quality Q8_0 to extremely low-quality IQ1_M, to meet the requirements of different hardware configurations and performance needs.

Natural Language ProcessingGguf

DeepSeek-V3.1 is a large language model developed by DeepSeek-AI and is an upgraded version of DeepSeek-V3. This model supports both hybrid thinking mode and non-thinking mode, and performs excellently in tool invocation, code generation, mathematical reasoning, etc., supporting a context length of 128K.

Natural Language ProcessingTransformers

This is the 4-bit quantized version of the DeepSeek-V3.1-Base model, converted using the mlx-lm tool and specifically optimized for Apple Silicon chips, providing efficient large language model inference capabilities.

Natural Language ProcessingMlx

DeepSeek-V3.1 is a large-scale language model developed by the DeepSeek team. It supports both thinking mode and non-thinking mode and performs excellently in multiple benchmark tests. It has powerful text understanding, generation, and reasoning capabilities.

Natural Language ProcessingTransformers

This is a quantized version of DeepSeek-V3-0324, processed using specific quantization methods, aiming to achieve more efficient use in different scenarios and supporting multiple quantization specifications to balance performance and efficiency.

Natural Language ProcessingTransformers

DeepSeek-R1T-Chimera is an open-source weights model that combines the intelligence of DeepSeek-R1 with the token efficiency of DeepSeek-V3.

Natural Language ProcessingTransformers

GLM-4-32B-0414 is a new member of the GLM family, featuring 32 billion parameters with performance comparable to the GPT series and DeepSeek-V3 series, supporting local deployment.

Natural Language ProcessingTransformersMultiple Languages

GLM-4-32B-0414 is a large language model with 32 billion parameters, comparable in performance to GPT-4o and DeepSeek-V3. It supports both Chinese and English, and excels in code generation, function calling, and complex task processing.

Natural Language ProcessingGgufMultiple Languages

GLM-4-32B-0414 is a new member of the GLM family with 32 billion parameters, offering performance comparable to GPT-4o and DeepSeek-V3, and supports local deployment.

Natural Language ProcessingTransformersMultiple Languages

DeepSeek-V3-0324 is a dynamically quantized version provided by Unsloth, supporting inference frameworks like llama.cpp and LMStudio.

Natural Language ProcessingGgufEnglish

A 9-billion-parameter large language model based on the DeepSeek-V3 architecture, trained from scratch using a fully open-source and exclusively English dataset of over 350 billion tokens, specifically designed for open-source community development and debugging.

Natural Language ProcessingTransformersEnglish

This is an experimental model that has not undergone complete training and is mainly used for research purposes. It is created through model fusion technology and requires a specific repair process to restore normal functionality.

Natural Language ProcessingTransformers

GLM-4-32B-Base-0414 is a new member of the GLM family. It has 32 billion parameters and is pre-trained on 15T high-quality data. Its performance can be comparable to that of advanced models such as GPT-4o and DeepSeek-V3. This model supports convenient local deployment and performs excellently in code generation, function calls, search-based Q&A, etc.

Natural Language ProcessingTransformersMultiple Languages

GLM-4-32B-0414 is a new member of the GLM family, a high-performance large language model with 32 billion parameters. This model was pre-trained on 15T of high-quality data, including a large amount of synthetic reasoning data, and performs excellently in multiple task scenarios such as code generation, function calls, and search-based Q&A. Its performance can rival that of larger-scale models like GPT-4o and DeepSeek-V3.

Natural Language ProcessingTransformersMultiple Languages