Llama-4-Maverick-17B-128E-Instruct-NVFP4 is a multilingual large language model that has undergone FP4 quantization processing. It is based on the Meta-Llama-3.1 architecture and is specifically designed for business and research purposes. By quantizing the weights and activations to the FP4 data type, this model significantly reduces the disk space and GPU memory requirements while maintaining good performance.

Natural Language ProcessingSafetensorsMultiple Languages

This is a quantized version of unsloth/Mistral-Small-3.2-24B-Instruct-2506. By quantizing the weights and activation functions to the FP4 data type, it reduces the disk size and GPU memory requirements while supporting vLLM inference. It has been evaluated on multiple tasks to compare the quality with the non-quantized model.

Natural Language ProcessingSafetensorsMultiple Languages

This is a quantized version of the Qwen3-235B-A22B-Instruct-2507 model. By quantizing the weights and activations to the FP4 data type, it significantly reduces the disk size and GPU memory requirements while maintaining performance similar to the original model.

Natural Language ProcessingSafetensorsMultiple Languages

This is the FP8 quantized version of the Meta Llama-3.1-8B-Instruct model. By performing FP8 quantization on weights and activation values, it significantly reduces the disk size and GPU memory requirements while maintaining good model performance.

Natural Language ProcessingSafetensors

This is the FP8 dynamic quantization version of the NVIDIA-Nemotron-Nano-9B-v2 model. Optimization is achieved by quantizing weights and activations to the FP8 data type, significantly reducing disk size and GPU memory requirements by approximately 50% while maintaining excellent text generation performance.

Natural Language ProcessingTransformersMultiple Languages

This is a quantized version of Qwen3-VL-235B-A22B-Instruct. By quantizing the weights and activation values to the FP8 data type, it effectively reduces the disk size and GPU memory requirement by approximately 50%. It supports text, image, and video inputs and outputs text, suitable for various natural language processing and multimodal tasks.

MultimodalSafetensors

Qwen3-VL-235B-A22B-Instruct-FP8-dynamic is a quantized version of Qwen3-VL-235B-A22B-Instruct. By quantizing weights and activations to the FP8 data type, it significantly reduces disk size and GPU memory requirements while maintaining high model performance.

MultimodalSafetensors

This is a speculative model based on the EAGLE-3 speculative decoding algorithm, specifically designed to work with the Qwen/Qwen3-32B model. It improves text generation efficiency through speculative decoding technology and performs well in mathematical reasoning and general Q&A.

Natural Language ProcessingTransformers

This is the INT8 quantized version of the Devstral-Small-2507 model. By quantizing the weights and activation values to 8-bit integers, it significantly reduces the GPU memory and disk space requirements by approximately 50% while maintaining good performance on coding tasks.

Natural Language ProcessingSafetensorsEnglish

Devstral-Small-2507-FP8-Dynamic is an optimized model obtained by quantizing the weights and activations of Devstral-Small-2507 to the FP8 data type, which can effectively reduce the GPU memory and disk size requirements by approximately 50% while maintaining performance close to the original model.

Natural Language ProcessingSafetensorsEnglish

Voxtral-Mini-3B-2507-FP8-dynamic is a quantized version of Voxtral-Mini-3B-2507, optimized for speech transcription, translation, and audio understanding, using the FP8 data type for quantization. This model has powerful audio understanding capabilities and can handle various audio-related tasks.

MultimodalSafetensorsMultiple Languages

This is a quantized version of the unsloth/gpt-oss-20b-BF16 model, using FP8 dynamic quantization technology, with efficient inference capabilities, and focusing on text generation tasks.

Natural Language ProcessingTransformers

Meta-Llama-3.3-70B-Instruct-NVFP4 is a quantized version of Meta-Llama-3.3-70B-Instruct. By quantizing weights and activations to the FP4 data type, it significantly reduces disk size and GPU memory requirements while maintaining high performance.

Natural Language ProcessingSafetensorsMultiple Languages

This is a quantized version of the Google Gemma-3n-E2B-it model, optimized by quantizing the weights to the INT4 data type, and can use the vLLM framework for efficient inference. The model supports audio-visual-text multimodal input and outputs text.

MultimodalTransformers

This is a quantized version model based on Google Gemma-3n-E2B-it. It uses the FP8 data type for weight and activation quantization, supports multimodal input of audio, vision, and text, and the output is text. The model achieves efficient deployment through vLLM, significantly improving inference efficiency while maintaining high accuracy.

MultimodalTransformers

This is a version of the Google Gemma 3N E4B IT model with FP8 dynamic quantization. By quantizing the weights and activation values to the FP8 data type, the inference efficiency is significantly improved while maintaining the performance of the original model. It supports multimodal inputs (text, images, audio, videos) and text outputs.

MultimodalSafetensorsMultiple Languages

The FP8 quantized version of the SmolLM3-3B model developed by RedHatAI. By quantizing weights and activation values to the FP8 data type, it significantly reduces GPU memory requirements and disk space usage while improving computational throughput.

Natural Language ProcessingSafetensorsMultiple Languages

This is a text generation model based on the EAGLE-3 speculative decoding algorithm, designed specifically for use with meta-llama/Llama-3.1-8B-Instruct. It improves text generation efficiency through speculative decoding technology and supports multiple languages.

Natural Language ProcessingSafetensorsMultiple Languages

A speculative decoding model specifically designed for Qwen/Qwen3-8B. It uses the EAGLE-3 algorithm to improve the efficiency and quality of text generation and obtains excellent generalization ability through training on multiple high-quality datasets.

Natural Language ProcessingTransformers

A large language model with 17B parameters based on the Llama-4 architecture. Optimized through INT4 weight quantization technology, it reduces the GPU memory and disk space requirements by 75%. It supports multilingual text generation tasks and can be efficiently deployed on the vLLM inference framework.

MultimodalSafetensorsMultiple Languages